Sunday, July 3, 2022
By:
Throughout this week, I continued my attempt to implement a reservoir computer training protocol involving a feed-forward neural network called Keras. This process has proven to be a substantial challenge. Although my IDE continued working this week without problems (hallelujah), I have been unable to accurately model the Van der Pol system with the Keras training strategy. Since all the work performed by Keras is happening behind the scenes, it’s difficult to discern what exactly is causing the drop in precision.
While the modeling is still not ideal, I am happy to report that I’ve gotten much closer than last week. The reservoir computer produces the correct periodic shape expected for the function and generally has the same amplitude. The main discrepancy lies in the wavelength and phase of the predicted signal. One major pitfall which likely augments the lack of synchronicity is the required level of precision in the output weight matrices. We are passing the mu parameters from the Van der Pol equation into the Keras deep learning routine and asking it to produce the output weight matrices for the reservoir computer (the matrices responsible for transforming the reservoir states into output data). At this point, the mean-squared-error between the correct matrix and the one produced by Keras falls on the order of 10^-6 over 500 values. This seems extremely close to the correct matrix, and in some sense indeed it is, but it’s not close enough to produce accurate predictions.
I made improvements this week by finding optimal dimensions and settings for the Keras network, often by running massive brute force loops which cycled through various settings and identified the ones which produced the lowest error (hence the title of this blog post). I also developed a unique normalization algorithm that scaled the signals based on their mu value and seemed to reduce error. In the next week, I’ll continue making adjustments to the code which I hope will result in the desired accuracy.
Outside of work, I once again enjoyed spending time with the other interns and exploring the city. On Wednesday, I went on a beautiful kayaking trip down the Potomac with Emma and Janessa. The event was hosted by Potomac Conservancy, an organization dedicated to cleaning the river running through DC, and the kayak rental fees went to their cause. On Thursday, I played Dungeons and Dragons with the other interns, so thanks to Taylor for DMing. On Friday, we walked down to Georgetown for an evening happy hour, and on Saturday we attended some Independence Day celebrations in the Dupont Circle area. I’m really looking forward to seeing some fireworks tomorrow on the Fourth of July, so don’t disappoint me DC. We’re in the nation’s capital, so you better have a serious show in the works.
Signing off til next week!
Benjamin Johnson