Basic battery monitoring ideas.

Wolf

Moderator
Joined
Sep 25, 2018
Messages
2,004
I need some assistance.

Now that I have my influxdb and grafana all set up I would like to have a device send my battery voltage to the influxdb via wifi.
I have a 48V nominal Pb battery bank that I would like to monitor.
I have looked, obviously not everywhere, but I can't seem to find a simple solution for this.
If anyone has a solution to monitor my battery bank with something like (RPi, arduino, or some other way) as to send this data to my influx db that would be highly appreciated. Once I get my powerwall set up with a BMS probably Batrium I won't need that anymorebut in the meantime what to do?
Additional sensors could include temp, humidity barometric pressure. Those I seem to be able to find but the Voltage one seems to elude me.
Any ideas?

Thanks in advance.
Wolf
 
This isn't off the shelf, but its not hard to use one of the wifi controller chips, aka ESP8266 chips to do this for you with a minimal number of external parts. A couple of the models have an analog pin for which you could use to monitor voltage via a simple resistor divider. Check out this: https://randomnerdtutorials.com/esp8266-adc-reading-analog-values-with-nodemcu/

Then it would be a matter of writing the code to read the value and send it upstream via wifi in some kind of timed loop I'd imagine.

FYI: Keep in mind these ESP8266 boards work at 3.3v and not the usual 5v arduino
 
I don't think there is an easy answer.

Your best bet is going to be either an arduino with an esp8266 to send data over WIFI or an ESP32 that has enough analog inputs for you to be able to use them. No matter what, you'll need to develop the hardware yourself from scratch and write all the code from scratch.
Well, you've got the majority of work left.

I've got my system up and running now - and trust me, you did the easiest part. Getting data into influxdb is the hard part. Good luck man. Check youtube there are a bunch of tutorials , all of which skip over some very important steps.
I'm working on some code at the moment. When I finish it, I'll have code for an esp8266, and an image of a pi to allow for all of this monitoring.

The issue with influxdb and grafana is that they aren't good for streaming data - as in, they are great to show data taken every couple minutes - but I want to know my voltage and temperature instantly. So I use MQTT for everything - but I have important information live streamed and displayed in Node-Red, and then once every five minutes it sends an AVG value via MQTT to Node-Red which then pushes it to InfluxDB and Grafana picks it up from there. You don't want to live-stream data to influxDB because then you'll just have way too much data in there - and if you use a PI, you'll destroy your storage medium fast.
 
The BMS described in
https://secondlifestorage.com/t-Good-Chinese-BMS-review

seem to have a serial output which they use to connect to PC and App, and as far as i have understood they have published the protocol in a terrible excel sheet.
Since you can see all cell voltages , and much more, that should be a possible basic unit for you.

a 13 S and 14 s system is available, but you have to order them as matching, they are not universal. 30 bucks.
 
Mikethezipper said:
I don't think there is an easy answer.
@ Mikethezipper

Well you wereright there wasn't an easy answer but.........
Over my holiday, (forSean's benefit) my Vacation I immersed myself into learning the Arduino IDE.
I was very successful.
Using an ESP8266 board I installed a BME280 i2cboard for environment stats and calculated a Voltage divider for 100V max to 3.3V for the A0 pin.
So far so good I got all the right outputs going to the serial monitor. Then Wifi and then sending data to the influx db.

image_ceqktm.jpg

image_xdrobm.jpg

The results in grafana

image_xfhzpv.jpg

I built 2 of the ESP8266 boards just to see the difference in V if there was any. .07V eh not bad for a 10 bit resolution
There is some noise in the measurements if you read the data by the secondso there are some solutions that I will try. There are suggestions to read the inputs twice in rapid succession and discard the first reading. Also a .01uf capacitor connected to pin A0.

Next step is to work with this code on an ESP32 and use all 8 available ADC inputs on the ADC1 SAR.ADC2 SAR is used by the Wifi so no go there for the other 10 ADC channels. If I need more ADC channels I can always use another ESP32 they are cheap enough.
The noise may be less noticable on the ESP32 as it has 12 bit resolution. At least I am hoping.
My next update on this I hope to have at leastmy 3 arrays and 2 batterybanks recording. Possibly I will throw some current readings in there also.

As far as Grafana is concerned the readings update fast enough for me as I am injecting data every 5 seconds. If the database grows to quickly I can always change the retention policy or slow the data feed down some.

Wolf
 
Did you get your IotaWatt posting to anything other than eMon yet Mr Wolf ?
 
Sean said:
Did you get your IotaWatt posting to anything other than eMon yet Mr Wolf ?

Oh yes actually the IotaWatt is posting to my locally hosted Influxdbhttp://wolftech.mynetgear.com:4562
You can log in as guest and password is guestw and look around.
The default dashboard is Westbrook Plant. There is also a Just Guages and the new one calledSensors
I also post to PVOutputhttps://pvoutput.org/list.jsp?id=74397&sid=65985
So yea the IotaWatt is posting to all 3.Emon is herehttps://emoncms.org/dashboard/view?id=53517for those interested.

Wolf
 
Wolf said:
Sean said:
Did you get your IotaWatt posting to anything other than eMon yet Mr Wolf ?

Oh yes actually the IotaWatt is posting to my locally hosted Influxdbhttp://wolftech.mynetgear.com:4562
You can log in as guest and password is guestw and look around.
The default dashboard is Westbrook Plant. There is also a Just Guages and the new one calledSensors
I also post to PVOutputhttps://pvoutput.org/list.jsp?id=74397&sid=65985
So yea the IotaWatt is posting to all 3.Emon is herehttps://emoncms.org/dashboard/view?id=53517for those interested.

Wolf
I dont seem to have the patience to get to grips with yet another graphing application - I have a good few years worth of PVOutput generation data online, and I see Iotawatt can post usage ... another rainy day project.

Have you seen the emonDC ? - there might be something you can glean for your project.

https://github.com/danbates2/emonDC


image_nqaqse.jpg
 
@ Sean,

Interesting!

I will keep an eye on this.
At this time though I am trying to get away from putting data out on the web. Just trying to keep it inhouse.
Plus its a new learning experience working with Arduino IDE and building my own sensor arrays. Giving my brain some
challenges is good.

Wolf
 
Mr Wolf,

Glad to see you got it up and running. Another possibility is to use your charge controller. What I'm actually going to do is to use an ESP8266 and RS485 to TTL converter to get data straight from my charge controller. Having voltage is nice, but it's only some of what's going on. Without also knowing how much power is going in or out, it's not the whole picture. For most uses though, voltage is enough to get an idea of what's going on - and knowing your voltage now is better than knowing your voltage and currents after another two months of development :p

I was going to use an ESP to do voltage monitoring until I realized that it's only got a 10 bit ADC, and even then it'll only tell me voltage. Also, the measurement is only as good as the reference voltage, which for that chip I don't think it's any good. Basically, I suspect it is susceptible to both temperature and voltage fluctuations into the esp chip... but I'm not sure.

As for how to get better readings - you can always take 10 readings and average them - but the noise in the readings will get averaged in too. What I've done in the past is to get say.. 10-20 readings, and take the median of those readings. That should give you the most accurate measurement of the voltage without temperature compensation.
 
Mikethezipper said:
Mr Wolf,

Glad to see you got it up and running. .............................

Mikethezipper,

Thanks

Well my cheap(actually inexpensive) MakeSkyBlue charge controllers do not have any kind of serial output. So till I step up to the plate and get a "real" charge controller, either Arduino or Rp or some derivative thereof is my option.

So just to be clear the ESP8266 was just an experimental first try. The results are encouraging and there is still a lot more work and coding to be done. Just getting the readings to post to the influx database was a challenge but the hurdle was overcome.
Also it seems a 0.1uf capacitor from the ADC pin to ground has seemed to stabilize the noise quite a bit.
Here is the Grafana 1 second sensor update and you can see after 19:00 hrs the sample rate smoothed right out after I installed the capacitor.

image_khsifq.jpg


I am now working on an ESP32S NodeMcu Dev board which allows me to have 6 ADC pins to my disposal. I also have hooked up an i2c BME280 and a VEML7700 LUX sensor. This board has 12bit resolution for the ADC so that should allow for more granular measurements. We will see.
As far as the chipsreference voltage is concerned yea I agree but there can always be an offset (int offset=xx)

I plan to measure Voltage on all 3 of my arrays, both battery banks, and with anAD8215 "OpAmp" I can measure my 100/75mv Shunt with reasonable resolution.
I'm not done yet I already have all the Shunts in place anyway feeding the standard meters most of us haveand will use a 2nd ESP32S with the AD8215to measure all the other shunts and dump that into influx.

image_rkpyvv.jpg


As far as other readings are concerned such as watts a simple calculation can be made in the code and sent to the database or grafana can do that too.
Additionally I have a lot of other data going into my influx from IotaWatt. I also have 2 wifi sticks on my GTILs that give me voltage, wattage etc. but I have to go to their website to get that data and I am trying to get everything local in one location.
Besides its fun.

Right now I am posting 10 readings to my influx database and I must say the ESP32S board is quite fast.Either that or I wrote some lean code by accident :p.
The ESP8266 is very slow in comparison.

Here are the results of posting just sample data with the ESP32S

image_tmnteh.jpg

The gap you see is me unplugging it from home and plugging it in at work.

image_znqexg.jpg


Wolf
 
OK Update on the ESP32 with all Arrays and Batteries hooked up.
The prototype is working good.
The ESP8266 is just there as a 2ndBattery 1 monitorfor now.

image_kywzwn.jpg


It's looking good on the chart also.
Don't pay attention to the Amperage chart that one is just feeding fake data I haven't set that up yet.
It looks like the BME280 had a glitch so theLux sensorat ~10:30AMwent ballistic so never mind that reading. :pBesides it's in the shed I will need to build an enclosure for it so that it can read the outside light intensity.


image_hjrxju.jpg


Wolf
 
Update:
The system is now complete.
1 ESP8266 running a BME280 for atmospheric info.
1 ESP32 with 1ADS1115 16-Bit ADC to do Voltages and 1 ESP32 with 2ADS1115 16-Bit ADCs to do the amperages.
They are working perfectly and it gives me a complete view of the plant now.
Since they are all i2c chips I will work on incorporating the whole thing into 1 package. Great stuff these ESP32 dev boards. Never had so much frustration and fun in one go.
I still need to add the amperage coming in from the arrays but the important data is now visually available

Wolf

image_cfevxc.jpg
 
Wow man, impressive work!
Just curious, if you are using an external ADC, why use 3 microcontrollers instead of just the ESP8266?
In case you are feeling extra good, it'd be nice if you shared your code on github or something and linked it here for the community to.

My original BMS project was going to go down the road you described, but I ended up going with the Texas Instruments BQ chip because those chips come pre-calibrated from the factory, with built in non-linearity and temperature compensation. Plus in the end they were a similar cost as a dedicated ADC chip alone.
I've finished most of the hardware aspect of it, but I'm slowly making my way over the the SW. Since we are using arduino based micros, we can probably share with eachother our code (my code is on github - look up mikethezipper)

Are you using the ESP chip or a raspi for the grafana server? I ended up using NodeREd vs influx&grafana because it doesn't actually save the data, just stores it in ram - so it's not using up lots of read/writes on the flash memory just to stream data.
 
Mikethezipper said:
Wow man, impressive work!
Just curious, if you are using an external ADC, why use 3 microcontrollers instead of just the ESP8266?
In case you are feeling extra good, it'd be nice if you shared your code on github or something and linked it here for the community to.

My original BMS project was going to go down the road you described, but I ended up going with the Texas Instruments BQ chip because those chips come pre-calibrated from the factory, with built in non-linearity and temperature compensation. Plus in the end they were a similar cost as a dedicated ADC chip alone.
I've finished most of the hardware aspect of it, but I'm slowly making my way over the the SW. Since we are using arduino based micros, we can probably share with eachother our code (my code is on github - look up mikethezipper)

Are you using the ESP chip or a raspi for the grafana server? I ended up using NodeREd vs influx&grafana because it doesn't actually save the data, just stores it in ram - so it's not using up lots of read/writes on the flash memory just to stream data.
Thanks Zip

Influx and grafana are hosted on ubuntu running on an old IBM bullet proof thinkpad soon to have 960gig SSD drive.
It all started with IotaWatt posting to influx and visualizing the data with grafana.
I have almost a year of IotaWatt data and it amounts to less than 80meg so I'm not worried about resources at the moment.

I initially was using just 1 esp8266 but since it only has 1 ADC I was limited to it and also 10bit resolution.
I kept the BME 280 on it as it was working just fine.
I wasn't thinking about the i2c at this time.
You have to remember I didn't even know what an Arduino was 8 month ago. If you had asked me I would have guessed a fancy Italian car manufacturer maybe.
Along comes the ESP32 with 6 usable ADC channelsand resolution was betterat12 bit.
Tried that and unfortunately readings where kind of all over the map. I was using twisted pair wire and all the right things but not really getting good results. Initial V divider was 47K and 1K to take a max of 160V to 3.3V well that was pushing the resistors wattage to the limit as my fingers discovered so Iwent all out to 1M and 22K and that worked.
Unfortunately the ESP32 was stilltoo jumpy with the results. I tried averaging and that seemed to settle things down some but in the long run the readings still were not satisfactory.
I was searching for a solution.
In comes the ADS1115 16bit gain programmable ADC. i2c communication life is good.Perfect.... great resolution for 3.3V and also for 75mV for the shunts.
Got 2 of them recognized on 1 ESP32 now just hook up the wires and go. Voltages were good and stable I was happy. OK hook the shunts up common ground on the low side 75 mV on the high side of the shunt what could go wrong?Ha I pulled a great noob the Amp readings where all over the place until I disconnected all but 1 and it was not so good either. I thought about it and soon realised with a common ground the whole wiring and every shunt would become a shunt so I was in essence shunting the shunts. Then I remembered reading about the ADS1115 being abler to do a differential inputbetween 2 channels. Aha the answer no need to use a common ground. So I reworked the Voltage ESP32 to do just voltage and since I have 6 or 7 ESP32s I just threw another one together
with 2 ADS1115s for the amp measurement off of the shunts. That all worked perfect.So that's how I came up with 3 microcontrollers.
Oh and by the way the ADS1115 does negative flow also so I can see if the battery is discharging which is a plus.But as I said in my post I will rework all this and put 4 ASD1115s on 1 ESP32 (compared to the ESP8266 the ESP32 is at least 10 times faster) chip and add the BME280 combine all the code and try it. But not right now.

I was looking at node red and mqtt etc. etc. till my head started to spin it was a challenge to get influx and grafana up and running let alone this thing about mosquitto and mqtt brokers ..........................?????
Up until a couple of month ago the most I coded was some batch files on a server.
So maybe you at some point can enlighten me on those avenues.

All in all coming this far in 4 months not bad for a newbie.

BTW I did a search on github for mikethezipper and i am a follower now.

Wolf
 
Update:

A full 24 hrs and all looks good.
The only thing I am noticing is that Array1 the voltage goes ballistic after midnight.
Then settles down in the morning.
As far as I know there is no UFO hovering over that array in the middle of the night. :p
Must be some kind of interference not sure what though.

Wolf

image_nnevem.jpg
 
Wolf said:
The only thing I am noticing is that Array1 the voltage goes ballistic after midnight.
Then settles down in the morning.
As far as I know there is no UFO hovering over that array in the middle of the night. :p
Must be some kind of interference not sure what though.

Are you sure there isn't a UFO? Do you have a camera to record that Array at night to make certain? Maybe your neighbor is going out there in the middle of the night and waving a flash light on it :p
 
Wow Wolf, I am so impressed I'm honestly humbled. You've gotten more done in a couple months than I have in a year :p

Since you are using an actual hard-drive, there is no benefit to anything I mentioned previously - that was only if you are using a raspi. Many users here use raspberry pi's , but they have a serious limitation in that they are basically really crappy computers that use sd-cards for memory that have a limited write cycle life.

Your issues with Array 1 are so high in voltage and last for so long that it makes me wonder what's going on too - it's not just noise, it's legit reading at double the voltage it should even be able to put out. Good luck man! I'm super interested to watch what you've been doing. Thanks for the updates and the inspiration to get off my butt and get something done
 
I doubt you gain much using the ESP32 in speed as such. If your software as such runs faster on the ESP32 you have coded it wrong :p

Yes the ESP32 is faster since it for instance have dual core but the task you perform should not be an issue. What could be is if you do alot of calculations or if you instead of running waiting for data from external sensors. That could halt the system

With that said the ESP32 is significantly better towards Wifi since it can use 1 core for that and the other for your normal tasks.

Good work and keep it up for sure!
The Arduino and ESP family is realy nifty tools to have and use. Its so easy to get data from them. The tricky part is to make sure the data gotten is accurate.

Influx is a good database for timeseries data. Just read up on how you save it because it can be done in many ways... Most of them wrong if you want to do some work on the data later on :p I have learned the hard way several times.
 
Thanks Zip and Daromer,

I'm so glad to have joined this board and forum to discuss ideas and be able to share successes and failures.

I basically was using the same code on the ESP8266 as I was on the ESP32 and my impression was that when I was doing a serial.println
to display the results on a serial monitor it seems that the ESP32 was writing the data faster to the influx database than the ESP8266. At least the Response:204 (write successful) would come back faster on the serial monitor.
daromer said:
The Arduino and ESP family is realy nifty tools to have and use. Its so easy to get data from them. The tricky part is to make sure the data gotten is accurate.
Very true. As I was bench testing all my stuff I would make sure that the readings I was getting were accurate thorough the normal ranges that the sensors where expected to have and adjusting the calculationsfor thatbefore I hooked everything up. After all we all know the expression "Garbage in Garbage out" :p

So that was my impression. The chips cost about the same so hey "whatever" it'snot like we are spending hundreds of dollars on them. :)

On another note yea I got three days recorded nowof this anomaly happening on Array 1.
If I had this high of a voltage it would have fried my ADS1115 as my voltage dividers are set up for a max of 3.3V at 160V.
I will try another ADS1115 but it's just odd it happens around the same time every night. and it's just Array1 :huh:

image_kxiczs.jpg



image_tylohp.jpg

Korishan
Are you sure there isn't a UFO? Do you have a camera to record that Array at night to make certain? Maybe your neighbor is going out there in the middle of the night and waving a flash light on it :p

LOL no I don't have a camera on the Array but if it is my neighbor with a flashlight it's got to be a zinger of one. Must be lighting up the whole neighborhood with 10 million candle power. I wonder if he is using Li-ion or Lipos?


Wolf
 
Back
Top