Wednesday, February 27, 2013

Simulation of a Geo Sensor

Hello,

here you can see my prototypical simulation of a geo sensor programmed with C#:


Group Box Sensor:
  • Description
    For specifying the current sensor. As I am working with weather data it is "humidity" here. 
  •  Sensor origin
    A file, where the data of the sensor is stored.
    In my example the file is very simple and locks like that:
    93
    94
    94
    91
    ...
  •  Interval
    Here you can specify the interval in seconds

Tab control "SVN":
  • Local Destination
    A file-name in an existing local repository, where the data should be stored
  •  Commit Message
    A message that will be stored for the SVN-commit

Tab control "Git":
  • Local Repository
    A path to a local repository
  • Local Destination
    A file-name in an existing local repository, where the data should be stored
  •  Commit Message
    A message for the SVN-commit
     
 
Import data in a Subversion repository:

When clicking the button "Start", data is imported from the geo sensor. The data is displayed in the view on the right side. The new data is then stored and committed to subversion.


Import data in a Git local repository:

When clicking the button "Start", data is imported from the geo sensor. The data is displayed in the view on the right side. The new data is then stored and committed to Git.


The same thing is possible in Java with the libraries "SVNKit" and "JGit". I already programmed two test programs with those libraries too. But I decided to stick to C#, because the later implementation of the visualization of the geo data will be done in C#.

Problems encountered:

I already encountered speed problems when working with SVN. I used an SVN repository from the University of Applied Sciences Salzburg. When I read the geo data like every second, the storage of the data in de SVN repository takes like 5 seconds. You can see the log message here:


It is no problem for the weather data which I am using, because that data comes in with a period of 5 minutes. But if you use faster sensors, five seconds for storing the data will be too long.
I don't have these problems with Git. But I have to mention, I store the data in the local Git repository, not in the remote one.

If you have any remarks or suggestions, feel free to comment :-)

Tanja

6 comments:

  1. I've been reviewing your initial proposal. I presume you know that the format of the data from the German weather station is roughly representative across the different geo-sensor systems available?

    Also, you are coding something here to generate random data? Are you storing the things you create in some versioning system? Can I check them out and run them?

    In terms of putting the data in the versioning system are you just storing in one big file? Adding a new line everytime a new piece of data comes in? or does each chunk of data get stored in a new file?

    What are the expected operations of the data:

    1) add new chunk of data?
    2) retrieve all data?

    I think what I could really use to understand all this is some specifications about the expected input and output of the system. What it has to do in order for it to be judged a success ...

    ReplyDelete
  2. Hi Sam,
    Thanks for your comment. Yes, I know that my weather data is just one example for geo data. The systems should be suitable for different kind of data.

    No, I don’t generate random data in my example. I read in data from a geo sensor and store it in a versioning system. That’s all.
    I am simulating the work of a geo sensor for further processing. I don’t have a real geo sensor, so I just read the data from an existing csv-file.
    My file is called "humidity.csv" and contains just the following content:
    ----
    93
    94
    95
    91
    92
    ----
    Each line contains a value for humidity.
    I just read the first line, store it in the versioning system
    Then after a defined interval I read the second line, store it in the versioning system and so on.
    I store all the data in one file. So if you check out the repository, you would just see a file like that above.

    This is just a helping system for the real one. I just simulate the geo sensor. I have to program a second software which can handle the data I get in the repository. But I am not so far at the moment.

    When my seconds program is ready, I want to start this simulation program several times for importing different types of data (e.g. temperature, humidity, rain, wind) for having a kind of real-time geo-sensor system.

    I think you are right, I should really start to make use cases for a better overall understanding.
    At the moment I am just simulating ... Maybe Thomas can say if this is representative for real geo sensor systems or not.

    ReplyDelete
  3. Thank you for the explanation - this makes a lot more sense. Where are the git and svn repos you are storing? Are they accessible to me? I'd love to browse them on google code or github ...

    ReplyDelete
  4. I stored the files I imported with Git just in my local repository.
    You can find the whole project here:
    git@github.com:tantschi/masterthesis.git
    For Subversion I am using a repository of my home university. It is secured with my student ID and a password. Is there any free public repository like GitHub for Subversion where I can store my stuff instead and give you access?
    But I think if you check out the project on Github, you won’t need access to the SVN-repository anymore.

    ReplyDelete
  5. Great thanks! Note that you can use html in these comments to great clickable hyperlinks https://github.com/tantschi/masterthesis. Also, you can get free svn code hosting from both http://code.google.com and I believe also from github. If you can switch to a public repo for the SVN I think that's to your benefit - and I'd love access to that too :-) Aha, and that github project link you provide that's the full code you are working on right? What about the repo for the data? Is that a separate repo on github?

    ReplyDelete
  6. Hi Sam,
    thanks for the hint. I didn't know that I am able to use html in my comments ;)

    That's the link to my SVN-repo of the data:
    http://malitz-thesis-svn.googlecode.com/svn/trunk/malitz-thesis-svn-read-only
    I created it with googlecode, because the github-svn-repo said "read only".

    That's the link to my github-repo of the data:
    https://github.com/tantschi/thesis_git.git

    But at the moment I commit just to the local repository of Git. I made the "pull" manually.

    ReplyDelete