Weather monitor

From EttiWiki
Revision as of 22:13, 27 March 2013 by Christian (talk | contribs) (→‎Filling Database: added predefined make rules)
Jump to navigationJump to search

The basic idea of the weathermon project is to obtain, record and monitor the weather at my home. All weather data sent from wireless sensors is to be stored in a database, from which different graphs are generated that can be monitored on a PC, over the Internet and on a smartphone.

System Overview

  1. Weather data is measured by sensors outside and inside the house. These sensors transmit all measured values via radio link.
  2. Transmitted values are received by a weather station and displayed for convenience.
  3. Transmitted values are also received by a small USB device which sends them to the PC connected.
  4. The PC reads received measurements and stores them in a database.
  5. From this database a web server serves all measurements to clients for display.

Sensors

The sensors and the weather station come as a set from ELV:

Receiver

For receiving the 868,35 MHz signals the following device from ELV is used:

Database

Data from weather sensors is stored in a Round Robin Database (RRDB) at different resolutions. This means, while high-resolution data is stored for the last 90 days, it also gets accumulated into daily data which is then kept for 10 years.

The database used is called RRDtool by Tobi Oetiker. It handles storing data at fixed intervals while updates come randomly, accumulating data and creating graphs. The latter is not used, instead a browser-based charting API is used to create dynamic charts.

Server

The server's job is to run the database and web server, so that clients can connect with a browser and retrieve and display all weather data.

  • The server is a standard Linux PC, but not a common desktop machine. Instead, a Raspberry PI is used.
  • The server runs an Apache Tomcat web server.

Interfaces

Protocol between Receiver and Server

The USB-WDE1 contains a CP210x serial-to-USB chip from Silicon Labs. Fortunately, Linux already contains a driver for that chip, so all that is needed is to plug the USB-WDE1 into the USB port. One can verify the driver is loaded correctly with lsmod: 

pi@raspberrypi ~ $ lsmod
Module                  Size  Used by
cp210x                 11783  0
usbserial              34545  1 cp210x

From then on the USB-WDE1 will transmit a serial ASCII string whenever a new measurement has been received. This can be verified with the following commands: 

stty -F /dev/ttyUSB0 9600
cat /dev/ttyUSB0

The ASCII string sent by the USB-WDE1 can look like this: 

$1;1;;;;;;-3,8;;;;;;;;94;;;;-4,2;98;0,0;36;1;0

Which means:

  • Temperature sensor address 5: -3.8°C
  • Humidity sensor address 5: 94%
  • Outside temperature: -4.2°C
  • Outside humidity: 98%
  • Wind speed: 0.0 km/h
  • Rain intensity: 36 rocker switch impulses (equals 8.8mm/m²)
  • Immediate rain detected

Protocol between Server and Clients

  • REST with JSON

Software Components

Filling Database

Serial data coming from the receiver must be interpreted and then stored into the database.

I wrote a short C program which basically does the following:

  • Open serial port and set communication settings.
  • Wait for and read a line from the USB-WDE1.
  • Split the semi-colon-separated values into separate strings and replace the decimal separator (, with .).
  • Call the rrd_update_r() function to enter these values into the RRDB.

The complete C code can be found here (TODO).

To compile it I use the following short Makefile (note that it makes heavy use of predefined rules). Using an extra Makefile was only necessary for including the librrd.so library. 

# predefined rule that compiles .c files into .o files: $(CC) $(LDFLAGS) n.o $(LOADLIBES) $(LDLIBS)
# predefined rule that links binary files from .o files: $(CC) -c $(CPPFLAGS) $(CFLAGS)

LDLIBS = /usr/lib/librrd.so

weathermon : weathermon.c

Querying the Database

For fetching data from the database I decided to use the java-rrd library. In contrast to other Java implementations for interfacing an rrdtool database, this has the following advantages:

  • It uses rrdtool itself, so it is 100% compatible with the database file format. Other implementations re-implement rrdtool but use a slightly different file format. As I use the original rrdtool to enter data into the database, the Java components must be compatible with it.
  • It doesn't use JNI framework, which would cause the software to be not platform-independent any more. As I develop under Windows but run the server with Linux, this was no option either.


The java-rrd library is written by Peter Stamfest. The original announcement can be found here:

Very helpful is the brief introduction by Manuel Aldana:


This library uses rrdtool's pipe mode: It calls "rrdtool -" which causes rrdtool to wait for commands from stdin and output its responses to stdout. This mode is also very fast, because the rrdtool instance only needs to be started once and from then on serves all requests.

The tricky part is to configure the place of the database file differently depending on which machine it runs on (development machine or server). I decided to use Tomcat's Context Parameters. I configured the context in META-INF/context.xml which resides in the deployable WAR file and is copied to a machine-specific directory upon first deployment. There I can modify it to fit the corresponding machine configuration. To access this context parameter in the source code I have to do the following: 

@javax.ws.rs.core.Context
public void setServletContext(ServletContext context) throws IOException {
  servletContext = context;
  String rrdbdir = servletContext.getInitParameter("rrdbdir");

REST Server

The web server implements a REST service which clients use to query data from the database. It is implemented in Java and uses the Jersey framework.

A very good introduction to implementing REST services with Jersey can be found here:

Basically, you only need to do the following:

  • setup the deployment descriptor (WEB-INF/web.xml)
  • implement the service class with using the correct annotations

The web.xml needs to contain the following information: 

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://java.sun.com/xml/ns/javaee" xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd" version="3.0">
  <display-name>weatherserv</display-name>
  <servlet>
    <servlet-name>Jersey REST Service</servlet-name>
    <servlet-class>com.sun.jersey.spi.container.servlet.ServletContainer</servlet-class>
    <init-param>
      <param-name>com.sun.jersey.config.property.packages</param-name>
      <param-value>at.ettisoft.weatherserv</param-value>
    </init-param>
  </servlet>
  <servlet-mapping>
    <servlet-name>Jersey REST Service</servlet-name>
    <url-pattern>/rest/*</url-pattern>
  </servlet-mapping>
</web-app>
  • The servlet container is implemented by Jersey, therefore the node web-app/servlet/servlet-class must point to com.sun.jersey.spi.container.servlet.ServletContainer.
  • This container expects the parameter com.sun.jersey.config.property.packages to point to the service classes at at.ettisoft.weatherserv
Retrieved from "http://proj.ettis.at/index.php?title=Weather_monitor&oldid=166"