Home > Web Site Updates > FOStation v1.0

FOStation v1.0

First off, you may ask why write another piece of weather station software when there are many fine ones available such as weewx, wview, pywws, & CumulusMX. Weewx and pywws are developed in Python and are very feature complete, while CumulusMX is for Windows but is no longer under active development. What I wanted was a low footprint Windows based application which simple stores the received weather records to a standard database. I also wanted my application to run as a Windows service and not bother with any UI infrastructure instead preferring to leave that to third party tools. Leaving the UI / reporting infrastructure out reduces the amount of code required massively. Instead you can develop custom reporting applications using existing mature technologies such as SQL Server Reporting Services. FOStation is designed with these goals in mind and should work with any Fine Offset compatible weather stations such as those manufactured by Aercus, Ambient Weather, DAZA, Elecsa, Fine Offset, Froggit, Maplin, Sinometer, Tycon & Watson. Personally I have a Aercus WS3083 weather station which I have tested FOStation with. The service is implemented with STL and ATL with no MFC dependencies with Visual C++ 2019.

Features

  • The service is implemented with STL and ATL with no MFC dependencies with Visual C++ 2019. The main application consists of just 1300 lines of C++ code. While running, FOStation uses less than 3 MB of working set memory.
  • Full source code for FOStation is included in the download for your perusal / education. Other modules of the author’s which FOStation depends on are mentioned below and should be downloaded and put into you FOStation source code directory if you want to rebuild FOStation for yourself.
  • The service has minimal runtime dependencies with the x86 binary coming in at 140 KB and the x64 binary coming in at 156 KB.
  • Prebuilt x86 and x64 binaries are provided in the download.
  • You will first need to download and install the Microsoft Visual C++ Redistributable for Visual Studio 2019 from https://visualstudio.microsoft.com/downloads/ for either x86 or x64.
  • To install the service, copy the exe in the ReleaseU (for the x86 UTF-16 binary) or ReleaseU64 (for the x64 UTF-16 binary) directories from the FOStation zip file to a directory where you want to install FOStation to and permanently run it from. Then type "FOStation.exe -install" from an Administrative Command Prompt in that directory. Internally FOStation uses the author’s CNTService framework and all the command line configuration values it exposes are available with the FOStation exe.
  • FOStation will log the data it collects to a SQL Server instance so you need to have some flavor of SQL Server installed somewhere which is reachable from the machine where FOStation is installed on. You could of course install SQL Server on the same machine as where you install FOStation. You can download the free SQL Server Express at https://www.microsoft.com/en-us/sql-server/sql-server-downloads. If you do not have SQL Server installed on the same machine as FOStation, then you will need to download and install the SQL Server drivers for either x86 or x64 from https://www.microsoft.com/en-us/download/details.aspx?id=56567 and install it on the machine on which FOStation will be running.
  • You might also want to download SQL Server Management Studio from https://docs.microsoft.com/en-us/sql/ssms/download-sql-server-management-studio-ssms. This tool allows you to easily manage your SQL Server instances using a GUI based utility. You should create a new blank database in your SQL Server instance called "WeatherStation" where you want to store the weather station data FOStation collects. Once FOStation is successfully collecting data, you can use this tool to review collected data, create ad-hoc queries against the data and develop customized reports, schedules and notifications using for example SQL Server Reporting Services. You can also of course develop custom functionality of your own against the database data FOStation collects.
  • The configuration values for FOStation which you will need to setup manually using the windows utility "Regedit" are as follows:
    • HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\FOSTATION\Parameters\General
      • LogFilenameSpecifier This is a REG_SZ value which contains the log filename which FOStation writes to. By default the value used is "FOStation.log". The log files written by FOStation will be located at C:\Users\"Registed Service Account name"\AppData\Local\FOStation\"LogFilenameSpecifier value". Note that this config value can have replaceable parameters as available from the strftime C runtime function. This mechanism allows you to implement daily log files for FOStation for example.
      • DBInitString This is a REG_SZ value which is the init string to use to connect to SQL Server for logging data. The default value is "DRIVER={ODBC Driver 13 for SQL Server};SERVER=.;DATABASE=WeatherStation;Trusted_Connection=yes;" Internally FOStation will use ODBC and an auto created stored procedure called "historicalDataUpsert" to write data to the database via an INSERT or UPDATE depending on whether a matching row already exists in the database table. You will need to change this value to suit where your SQL Server instance is available as well as any other database init values which you need.
      • StationType This is a REG_DWORD value. Setting this to 0 which is default means "With Solar Data", while 1 means "Without Solar Data". Fine Offset compatible weather stations store there data in two formats. The first record format is 16 bytes long and does not contain solar data (i.e. UV and illuminance data) while the second format is 20 bytes long and does include solar data. You will need to specify the correct type for your particular weather station device type. Please note that data that is not collected such as UV and illuminance when this config value is 1 will result in null values being written to the database for these columns.
      • MaxHistoricalRecords This is a REG_DWORD value that corresponds to the maximum number of historical records to process. The default value is 0 which means no historical records will be processed. Fine Offset compatible weather stations store historical records in a ring buffer on the indoor console. Setting this config value to a non-zero value will allow these historical records to be read and logged to the database before the service starts processing live records from the weather station. The maximum number of historical records stored is 3264 for weather stations which do not collect solar data and 4080 for stations which do collect solar data.
      • DeviceTimeout This is a REG_DWORD value which allows reads and writes to the weather station to use timeouts. Fine Offset weather stations are known to suffer from a USB lockup bug which seems to occur when USB activity occurs at the same time when the console is reading sensor events from the outdoor station or when the interval ring buffer is being updated. By default this value is 8000 which represents 8000 milliseconds. If this value is set to a non-zero value then the reads and writes will internally use Win32 Overlapped IO and will timeout if reads or writes take longer that the config value. Entries will be written to the FOStation log files to indicate that the weather station console will need to be reset by taking out its batteries when read or write timeouts are detected. If you use a value of 0, then synchronous reads and writes will be used. Internally FOStation will use the author’s HIDWrappers classes to talk to the weather station console’s USB port via the Windows HID APIs.
      • DebugAuditing This is a REG_DWORD value and if this value is set to 1, then extra entries will be written to the FOStation log files for diagnostic / debugging purposes. The default is 0 for this value.
      • METWOWSiteID This is a REG_SZ value which FOStation uses to identify itself to the British Met Office WOW. This number appears (in brackets) next to or underneath the name of your site on the WOW site information page. Internally the FOStation service uses the author’s WinHTTPWrappers and CVersionInfo classes to support this functionality.
      • METWOWSiteAuthenticationKey This is a REG_SZ value and is the second item you require for logging to the British Met Office WOW. This is a 6 digit number that you specify when setting up a WOW account and is used to ensure data is coming from you and not another user.
      • DBLoggingInterval This is a REG_DWORD value and is the minimum logging interval in minutes to log to the database. Please note that Fine Offset weather stations will log an updated record to the indoor console at a configurable interval as specified by a byte stored at address 16 in the fixed block on the console (please see http://www.jim-easterbrook.me.uk/weather/mm/ for details on the memory layout of Fine Offset weather stations). You can also use the CFOStation class in the FOStationWrappers.h module included in the FOStation download and the CFOStation::WriteAddress method to change this value. For example I wrote some throw away code while developing FOStation to call "device.WriteAddress(16, 1)" to change the collection period to 1 minute for my weather station. FOStation will monitor the indoor console for updated records and when it detects a new record available, will trigger a read. This config value can be used to write to the database at a rate different to the rate at which the weather station produces new records. The default is 5 minutes for this value.
      • METWOWLoggingInterval This is a REG_DWORD value and is the minimum logging interval in minutes to log to British Met Office WOW web site. The default is 15 minutes for this value.
      • AvoidReadPeriod This is a REG_DWORD value. This value is the number of seconds either side of the station clock and sensor clock times which the FOStation service determines during its initial synchronization process. The FOStation service includes quite complicated internal logic to avoid reading from the station "AvoidReadPeriod" seconds either side of these times. The default is 3 seconds for this value. During testing of the service over a period of a couple of weeks, the author was unable to get USB lookups to occur during normal operations of the FOStation service. On the other hand, the author was able to get a couple of spurious lookups to occur if you just tried to continually read from the weather station in a tight loop, so these additional precautions which FOStation takes when reading from the weather station are necessary. The synchronization process is rerun every 24 hours internally in the FOStation service to avoid the computer clock getting too much out of sync with the clock on the weather station console.
    • HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\FOSTATION\Parameters\LastRainfall
      • Count This value is written and read internally by FOStation and should not be edited by the end-user. This is used to store the last rainfall counter value read from the weather station. This is used to ensure correct values are written to the "rainfall" column in the database upon service or computer restarts.
    • HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\FOSTATION\Parameters\MidnightRainfall
      • Count, Year, Month Day These values are written and read internally by FOStation and should not be edited by the end-user. They are used to store the rainfall counter value for the last midnight read from the weather station. This is used to ensure correct values are written to the "rainfallSinceMidnight" column in the database upon service or computer restarts. The Year, Month and Day values are used to store the date for the midnight count while the Count value stores the actual numeric count.
    • On the initial run of the FOStation service, a table will be auto created if it does not already exist in the configured database where the weather records will be stored. The schema for this table is:
      CREATE TABLE [historicalData]
      (
        [dateTime] [datetime] NOT NULL,
        [indoorHumidity] [int] NULL,
        [indoorTemperature] [float] NULL,
        [outdoorHumidity] [int] NULL,
        [outdoorTemperature] [float] NULL,
        [pressure] [float] NULL,
        [avgWindSpeed] [float] NULL,
        [gustWindSpeed] [float] NULL,
        [windDirection] [int] NULL,
        [rainfallSinceMidnight] [float] NULL,
        [rainfall] [float] NULL,
        [totalRainfall] [float] NULL,
        [UV] [int] NULL,
        [illuminance] [float] NULL,
        CONSTRAINT [PK_historicalData] PRIMARY KEY CLUSTERED
        (
          [dateTime] ASC
        )
      )
      • dateTime is the local time when the record was stored.
      • indoorHumidity is the humidity from the indoor console of the weather station as a percentage.
      • indoorTemperature is the temperature from the indoor console of the weather station in degrees Celsius.
      • outdoorHumidity is the humidity from the outdoor station as a percentage.
      • outdoorTemperature is the temperature from the outdoor station in degrees Celsius.
      • pressure is the pressure from the outdoor station in hectopascals. The measurement is with respect to the stations altitude and not with respect to mean sea level.
      • avgWindSpeed is the average wind speed in m/s for the captured recording interval from the outdoor station.
      • gustWindSpeed is the maximum wind speed in m/s for the captured recording interval from the outdoor station.
      • windDirection is the direction of the wind. 0 represents due North, 90 due East and so on.
      • rainfallSinceMidnight is the amount of rainfall which has occurred to this point from midnight local time to "dateTime" in millimeters.
      • rainfall is the amount of rainfall in millimeters which has occurred between the last time a record was logged to the database and the current record’s "dateTime".
      • totalRainfall is the total amount of rainfall ever accumulated by the station since it was turned on. Internally the FOStation code will handle rollovers of the rain counter into this value.
      • UV is the UV value recorded. 0 represents None up to 5 which means Extreme.
      • illuminance is the measure of light intensity for the outdoor station in lux. A completely dark night would correspond to 0, twilight corresponds to roughly 400 while 10,000 upward represents bright daylight.
      • Note that FOStation includes code to detect empty data in records read from the weather station and these will end up being stored as null columns in the database.
  • Once you have the service installed and the appropriate configuration values setup, you should be able to start the FOStation service using the standard Windows "Computer Management -> Services and Applications" management tool. You should check the contents of the FOStation log file to make sure startup of the service occurred ok. Upon starting, the FOStation will take a few minutes to synchronize with the weather station so that it can minimize reading from the weather station at critical times to avoid the USB lookup bug. These details as well as the config values being used and the other logic which FOStation does during startup will be logged to the FOStation log files. A successful startup of the service should generate something like the following in the log files (assuming you have set DebugAuditing to one):
    31/08/2019 22:52:00 ******* Service starting *******
    31/08/2019 22:52:00 Station type: 0
    31/08/2019 22:52:00 LogFilenameSpecifier: FOStation.log
    31/08/2019 22:52:00 DBInitString: DRIVER={ODBC Driver 13 for SQL Server};SERVER=.\SQLEXPRESS;DATABASE=WeatherStation;Trusted_Connection=yes;
    31/08/2019 22:52:00 AvoidReadPeriod (seconds): 3
    31/08/2019 22:52:00 MaxHistoricalRecords: 0
    31/08/2019 22:52:00 DeviceTimeout (milliseconds): 8000
    31/08/2019 22:52:00 DebugAuditing: 1
    31/08/2019 22:52:00 MET WOW Site ID: XXXXXXYYYYZZZ
    31/08/2019 22:52:00 MET WOW Site authentication key: ******
    31/08/2019 22:52:00 MET WOW logging interval: 15
    31/08/2019 22:52:00 DB Logging interval: 5
    31/08/2019 22:52:00 Product Version info: 1.0.0.0
    31/08/2019 22:52:00 Reloading rainfall count of 2508 for today
    31/08/2019 22:52:00 Reloading last rainfall count of 2516
    31/08/2019 22:52:00 Enumerating Fine Offset weather station devices connected to this computer
    31/08/2019 22:52:00 Found Find Offset weather station with device path of \\?\hid#vid_1941&pid_8021#9&fb18f1f&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}
    31/08/2019 22:52:00 Opening device ‘\\?\hid#vid_1941&pid_8021#9&fb18f1f&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}’
    31/08/2019 22:52:00 Creating ODBC environment
    31/08/2019 22:52:00 Creating ODBC connection
    31/08/2019 22:52:00 Connecting to database, initialization string: DRIVER={ODBC Driver 13 for SQL Server};SERVER=.\SQLEXPRESS;DATABASE=WeatherStation;Trusted_Connection=yes;
    31/08/2019 22:52:01 Connected to database
    31/08/2019 22:52:01 Creating ‘historicalData’ table in database if necessary
    31/08/2019 22:52:01 Creating ‘historicalDataUpsert’ stored procedure in database if necessary
    31/08/2019 22:52:01 Synchronizing with weather station (this may take a few minutes)
    31/08/2019 22:52:26 Contents of current write address of 0xF290 has changed, determined sensor clock time to be 31/08/2019 22:52:26
    31/08/2019 22:53:20 Current write address has changed from 0xF290 to 0xF2A4, determined station clock time to be 31/08/2019 22:53:20
    31/08/2019 22:53:20 Determining historical record data available
    31/08/2019 22:53:20 Current location address: 0xF2A4
    31/08/2019 22:53:20 Determined next expected write time will be 31/08/2019 22:54:20
    31/08/2019 22:53:20 Historical records available: 3264
    31/08/2019 22:53:20 Historical records to be read: 0
    31/08/2019 22:53:20 Completed reading historical data
    31/08/2019 22:53:20 Historical records to be stored: 0
    31/08/2019 22:53:20 Completed storing historical data
    31/08/2019 22:53:20 Skipping read because not safe to do so at this time
    31/08/2019 22:54:01 Skipping read because not safe to do so at this time
    31/08/2019 22:54:21 Skipping read because not safe to do so at this time
    31/08/2019 22:54:31 Processing new data at address 0xF2B8
    31/08/2019 22:54:31 Processing record at address 0xF2B8 for 31/08/2019 22:54:31
    31/08/2019 22:54:42 Saving record to database
    31/08/2019 22:55:27 Remembering rainfall count of 2516 for 31/08/2019 22:54:31
    31/08/2019 22:55:27 Uploading record to MET WOW
    31/08/2019 22:55:39 Skipping read because not safe to do so at this time
    31/08/2019 22:55:49 Processing new data at address 0xF2CC
    .
    .
    .
  • Once the service is running correctly, you might to change the startup type on the service from "Manual" to "Automatic" or "Automatic (Delayed Start)" so that the service will auto restart whenever the computer is rebooted.
Advertisements
Categories: Web Site Updates
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: