Smart/predictive home energy management system?

Has anyone tried building a smart/predictive home energy management system? As a human withreasonable intelligence, I can easily predict days aheadwhen my batteries will run dry, when theyare likely tooverflow,if I should switch off the powerwall in the eveningto have enough energy the following rainy day,if I should delaychargingthe PHEV or just usenightgrid power, etc.I'm thinking it's totally doable to have a raspberry manage it optimally for me, perhaps usingAI /machine learning, but I'm not quite sure what libraries/packages are available to get me started.

It would have the followinginput data:
* Hourlyenergy generation data for past year
* Hourly energy consumption data for past year
* 2 days ofhourly weather forecast of reasonable accuracy
* 10 days of rough weather forecast of questionable accuracy
* Real time power into/out ofthe grid
* Real time charge status of my Powerwall
* Expensive grid rate between 7am~11pm, cheap night power between 11pm ~ 7am
And various remote relays can be controlled to switch on/offthe powerwall,EV charging, hot water heater, well pump, etc.

I run an RPi here, more for monitoring, not load switching.
Daromer has a complex setup running too & an iso you could look at.
Also have a look at "solar spy", discussion by author here: https://whrl.pl/RfmB5g he's doing some predictive stuff, it takes a while to load.

You might also have it log/calculate:
- typical seasonal solar kWhrs input expected eg adjust for spring/summer/autumn/winter sun hours
- energy so far in today
- typical base loads & values of switchable loads
- hot water tank top & middle temps
You could use crontasks & php
Using a database you could run a Predicted & Actual available kWhrs numbers updated every few mins.
Bit of calculation you could prioritize (or cycle through, etc) switching on/off + adjust run times of larger loads.
Make it display on a web page with a few buttons for easy adjustments, eg top ups, "run the EV charger tonight on cheap off peak", "charge the powerwall tonight from cheap off peak", etc.

Thanks, but looking at the "solar spy" source code, it basically relies on data from 2 external web sites to estimate his/her PV production... I want a lot more than that.

For humans, it's just basic intuition, but there are so many aspects(parameters) for a program to consider to estimate both production and consumption(s). Eg. "typical base load" varies between 500W and 3000W, depending on the season (sunrise/sunset), day of the week, time of day, the outside temperature, sunshine, stored heat in the walls, wind speed, humidity (for A/C), etc. My traditional approach would be to figure out each component individually, db lookup / interpolate from past data, weigh their contribution and put out one final number.
Solar production estimation is equally complicated in my case as I have 9 arrays of varying sizes and angle. And I have one huge tree that throws a travelling shadow late in the afternoon, depending on the season.

I was hoping that there'd be a smarter, more automated machine learning system that I can just dump all the past data on (incl historical weather data), and it'll figure out the relations by itself, and make predictions given a weather forecast. Most importantly, continually learning as it compares predictions vs actual result.

I haven't heard of a system that does this all for you.
I think Tesla's battery management algorithm does some of this but it'll be closed source, etc. Might help to read up on what/how it does that.
Overall, my thinking is there are too many different systems, manufacturers & models out there for a one-size-fits-all solution commercially.
Similarly open source & DIY tend to make something specific to their needs/gear.
Seems like you might have to dive in & DIY!

I've found something interesting:

TensorFlow, by the Google Brain Team
https://www.tensorflow.org/tutorials/estimator/boosted_trees

The example feeds (teaches) the system the Titanic passenger data (survived or not, age, sex, fare, class, deck, etc), and then makes a survival prediction for a [hypothetical] passenger that was not on the teaching list.
I think I can adapt this by feeding my past generation data (kWh produced, date/time, weather, temperature), and then let it make a prediction using the weather forecast.
Do the same for consumption, and I'm 90% where I want to be.