Exponential filter

Support forum for MegunoLink
Post Reply
steve.
Posts: 3
Joined: Sun Mar 08, 2020 8:53 am

Sun Mar 08, 2020 9:07 am

Hi all,
My first post to the forum and apologies if this is the wrong board to post this. I have been using the Megunolink exponential filter on an ultrasonic tank level application. I am using three filters, one for distance, one for temperature corrected distance and one for level. The filters work well but I want to initialise them with the last recorded value rather than a constant so that when the board restarts for any reason, they take the initial value in the constructor from the last recorded value. I am using Blynk as the interface app and the distance and level values are written to and stored on the Blynk server.
As part of the sketch initialisation, I call these stored values, assign them to start variables and then initialise the filters with these variables. When the sketch starts, however, the filters are initialised with zero. I have written one of the start variables (the last recorded level filter value) to the serial monitor and I can verify that the correct last recorded value is being pulled from the server. I have attached the sketch and would certainly appreciate any thoughts.

Code: Select all

/* Comment this out to disable prints and save space */
#define BLYNK_PRINT Serial


#include <ESP8266WiFi.h>
#include <BlynkSimpleEsp8266.h>
#include <Filter.h>
#include <TimeLib.h>
#include <ArduinoOTA.h>
#include <DNSServer.h>            //Local DNS Server used for redirecting all requests to the configuration portal
#include <ESP8266WebServer.h>     //Local WebServer used to serve the configuration portal
#include <ESP8266mDNS.h>
#include <ESP8266HTTPUpdateServer.h>
#include <WidgetRTC.h>
#include <OneWire.h>
#include <DallasTemperature.h>



// Define pins and assign variables
#define oneWireBus 14   //Define temperature probe connected to pin 12 (D6) 
#define trigPin  12  // Arduino pin tied to trigger pin on the ultrasonic sensor 13 D7).
#define echoPin  13  // Arduino pin tied to echo pin on the ultrasonic sensor 14 (D5).

//Define variables
int MAX_DISTANCE = 300; // Maximum distance we want to ping for (in centimeters). Maximum sensor distance is rated at 400-500cm.
int TANK_FULL_HT = 202;  // Height of liquid at 100% full in cm
int TANK_EMPTY_HT = 10;   //Height of unusable liquid at base of tank in cm
int SENSOR_DIST = 26;    //Distance from the sensor to the liquid at 100% full in cm
int TANK_DIA  = 1003;     //Tank diameter in cm
int TANK_FULL_VOL = 161892; // Tank volume at 100% in litres
long duration;  // For Ping sensor
float distance;  //distance calculated from ultrasonic sensor
float distanceCorrected;  //Temperature corrected distance
float levelFilt;  //current value of level from exponential filter
float temp;
float speedSound;
float LEVEL;
float levelStart;  //variable to hold the last recorded level value (from Blynk Server).  Used to initialise exponential filter
float distStart;  //variable to hold the last recorded distance value (from Blynk Server).  Used to initialise exponential filter
float distCorrStart;  //variable to hold the last recorded temperature corrected distance value (from Blynk Server).  Used to initialise exponential filter


// You should get Auth Token in the Blynk App.
// Go to the Project Settings (nut icon).
char auth[] = "xxxxxxxxxxxxxxxxxxxxxxxxxxx"; //code for development board


// Your WiFi credentials.
// Set password to "" for open networks.
const char* host = "tank_level";
char ssid[] = "xxxxxxxx";
char pass[] = "xxxxxxxxxx";

// Mac address should be different for each device in your LAN
//byte arduino_mac[] = { 0x39, 0x7D, 0x11, 0xB4, 0x40, 0x7A };
IPAddress arduino_ip ( 192,  168,   1,  117);
IPAddress dns_ip     (  8,   8,   8,   8);
IPAddress gateway_ip ( 192,  168,   1, 101);
IPAddress subnet_mask(255, 255, 255,   0);

ESP8266WebServer httpServer(80);
ESP8266HTTPUpdateServer httpUpdater;



//Define onewire instance and pass to Dallas Temperatuure
OneWire oneWire(oneWireBus);
DallasTemperature sensors(&oneWire);

//Widget real time clock
WidgetRTC rtc;

BLYNK_CONNECTED() {
  Blynk.syncVirtual(V1, V3, V6);  //Sync the start values with the Blynk server value.  V1 is corrected distance, V3 is distance, V6 is calculated level
  rtc.begin();
}

BLYNK_WRITE(V1) { //calls the value stored on the Blynk Server
  distCorrStart = param.asFloat(); //last recorded value of corrected distance
}

BLYNK_WRITE(V3) { //calls the value stored on the Blynk Server
  distStart = param.asFloat();  //last recorded value of distance
}


BLYNK_WRITE(V6) { //calls the value stored on the Blynk Server
  levelStart = param.asFloat();  //last recorded value of level
  Serial.print("Level start =");  //check serial monitor to see if server value being called
  Serial.println(levelStart);
}

// Define exponential filters for distance and tank level
ExponentialFilter<float> distFilter(5, distStart);  //initialise distance filter with last recorded value
ExponentialFilter<float> distFilterCorrected(5, distCorrStart);  //initialise corrected distance filter with last recorded value
ExponentialFilter<float> lvlFilter(5, levelStart);  //initialise level filter with last recorded value


BlynkTimer timer;

void setup()
{
  // Debug console
  Serial.begin(115200);


  // Define pin modes for the trigger and echo pins of ultrasonic sensor
  pinMode(trigPin, OUTPUT);
  pinMode(echoPin, INPUT);

  // Hostname defaults to esp8266-[ChipID]
  ArduinoOTA.setHostname("TankLevel");

  WiFi.config(arduino_ip, dns_ip, gateway_ip, subnet_mask);
  WiFi.begin(ssid, pass);

  if (WiFi.status() != WL_CONNECTED) {
    WiFi.begin(ssid, pass);
  }


  while (Blynk.connect() == false) //wait for Blynk to connect before proceeding
  {
    Blynk.config(auth);
  }


  MDNS.begin(host);
  sensors.begin();

  httpUpdater.setup(&httpServer);
  httpServer.begin();

  MDNS.addService("http", "tcp", 80);
  Serial.printf("HTTPUpdateServer ready! Open http://%s.local/update in your browser\n", host);


  ArduinoOTA.begin();
  Serial.println("Ready");
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());

  // Setup a Blynk timer function to be called every two seconds
  timer.setInterval(2000L, myTimerEvent);
}

// This function calculates and sends tank level to Blynk every 5 seconds
void myTimerEvent()
{
  // Send pulses to ultrasonic sensor
  digitalWrite(trigPin, LOW);
  delayMicroseconds(2);
  digitalWrite(trigPin, HIGH);
  delayMicroseconds(20);
  digitalWrite(trigPin, LOW);

  // End Pulse & Calculate distance
  duration = pulseIn(echoPin, HIGH);

  distance = (duration / (2)) / 29.1; //calculate distance from time measurement from ultrasonic sensor
  distFilter.Filter(distance);  //apply filter to distance measurement

  // Get temperature to enable distance correction
  sensors.requestTemperatures();
  temp = sensors.getTempCByIndex(0);

  //Calculate corrected distance based on temperature correction
  speedSound = 331.3 + (0.606 * temp);
  distanceCorrected = (duration / (20000.0)) * speedSound;
  distFilterCorrected.Filter(distanceCorrected);  //apply filter to corrected distance measurement

  LEVEL = (((TANK_FULL_HT - TANK_EMPTY_HT) - (distance - SENSOR_DIST)) * 100) / (TANK_FULL_HT - TANK_EMPTY_HT);
  lvlFilter.Filter(LEVEL);  //apply filter to level calculation
  levelFilt = lvlFilter.Current();

  if (LEVEL > 100) LEVEL = 100;
  else if (LEVEL < 0) LEVEL = 0;

  // Send data to Blynk
  Blynk.virtualWrite(V3, distFilter.Current()); //sends current filter value of distance to Blynk server
  Blynk.virtualWrite(V1, distFilterCorrected.Current());  //sends current filter value of corrected distance to Blynk server
  Blynk.virtualWrite(V5, lvlFilter.Current());  //sends current filter value of level to Blynk server
  Blynk.virtualWrite(V0, distance);  //sends current raw value of distance to Blynk server
  Blynk.virtualWrite(V27, LEVEL);  //sends current raw value of level to Blynk server
}


void loop()
{
  Blynk.run();
  timer.run(); // Initiates BlynkTimer
  ArduinoOTA.handle();
  httpServer.handleClient();
}
Paul
Posts: 22
Joined: Wed Jun 10, 2015 10:35 pm

Mon Mar 09, 2020 9:06 am

Hi Steve,

The line ExponentialFilter<float> distFilter(5, distStart); //initialise distance filter with last recorded value
runs before your void start(){...} function. At this point the program hasn't retrieved anything from the Blynk server. So the value of distStart will be 0.

I think what you need is something like this, for example:

Code: Select all

BLYNK_WRITE(V3) { //calls the value stored on the Blynk Server
  float distStart = param.asFloat();  //last recorded value of distance
  distFilter.setCurrent(distStart);
}
You also don't need distStart as a global variable and don't need to pass it to the distFilter constructor.

I'm not completely sure how the Blynk.syncVirtal function and the BLYNK_WRITE(...) work, so this is a bit of a guess. I'm assuming some how the BLYNK_WRITE(...) function runs when the device is connected.

Let us know how you get on,
Good luck,
Paul.
steve.
Posts: 3
Joined: Sun Mar 08, 2020 8:53 am

Mon Mar 09, 2020 11:34 am

Hi Paul,
Many thanks for the prompt response. I tried your suggested code mod and got it to work but I also had to move the Blynk_Write commands after the constructor to prevent a compile error as the filter hadn't been initialised when the setCurrent call was made.

Many thanks for your help. My only other question relates to your comment that the variables don't need to be included in the constructor. What is the format for the constructor in this case? I still want to set a weight and tried

Code: Select all

ExponentialFilter<float> lvlFilter(5);  
but it returned a compile error. Is it a matter of including a line to set the weight when I set the current value for the filter? The code would look something like this

Code: Select all

BLYNK_WRITE(V5) { //calls the value stored on the Blynk Server
  levelStart = param.asFloat();  //last recorded value of level
  lvlFilter.SetCurrent(levelStart);
   lvlFilter.SetWeight(5);
  Serial.print("Level start =");  //check serial monitor to see if server value being called
  Serial.println(levelStart);
}
Cheers
Steve
Paul
Posts: 22
Joined: Wed Jun 10, 2015 10:35 pm

Mon Mar 09, 2020 7:05 pm

Hi Steve,

Glad it worked out.

To initialize the filter variable you still need an initial value, just don't need the variable for it. So:

Code: Select all

ExponentialFilter<float> lvlFilter(5, 0);
Then you won't need to set the weight in the BLYNK_WRITE function.

It is worth thinking about the race condition too. If your program sends a value to the Blynk server before the BLYNK_WRITE retrieves the old value then you'll be initializing the filter with the wrong value. You could include another global variable that is set to true when you've gotten the initial value:

Code: Select all

bool InitializedFilter = false; 

BLYNK_WRITE(V5)
{
  InitializedFilter = true;
  // etc
}

void loop()
{
  if (IninitializedFilter == true)
  {
    // update filter value. 
  }
}
I'm not sure what happens the very first time you run the programme. The Blynk server might not have an initial value then. So the filter never gets initialized and it never has an initial value. You could solve this with a timeout that starts the filter (sets InitializedFilter to true) if a response isn't received from the server. Or you could store previous filter values in the device eeprom and initialize the filters from there. If you use the device eeprom it could be worth storing the filter value when it has changed a lot (e.g. 5%) to reduce wear on the eeprom.

Have a good day
Paul.
steve.
Posts: 3
Joined: Sun Mar 08, 2020 8:53 am

Tue Mar 10, 2020 10:39 am

Hi Paul,
Many thanks for your feedback and suggestions. Has definitely helped me nail this one and is working as I want. My intention for wanting this was to ensure that the level and distance trends don't contain a large dip or spike when the board resets and the filter initial value is very different from the last recorded. The trend picks up from where it left off. There is a very small difference between the server stored value and the initialisation values but not worth chasing down.
In terms of Blynk and the server, I'm no expert but my understanding is this - Blynk has a virtual pin feature whereby data can be assigned to a virtual pin and written to the server where the data can be connected with the app for display. This server data is not erased when the board restarts and I have come to think of it as virtual eeprom (in fact some users use the server values in lieu of eeprom in their sketches). The trick then is to retrieve these values and this is the function of Blynk_syncVirtual, which executes a call to the highlighted virtual pin Blynk_Write function, and the Blynk_Write function which calls the current stored value with the param.asFloat() code.
Hence the starting values can be initialised with the server values before setup or the main program is run.

Thanks again. This is a very nice filter with some good features.

Steve
Paul
Posts: 22
Joined: Wed Jun 10, 2015 10:35 pm

Wed Mar 11, 2020 8:34 am

Great. Glad you got it worked out.
Post Reply