To use a web "framework" or not - results hosting

[Harvs]

For a project of mine I've been collecting data from a uC (and will continue to), and using gnuplot with python to produce nice looking graphs of the results (thanks to whomever it was on here that put me onto that one.)

I need to take that one step further and host a dynamic webpage to display graphs of sections of the data based on user form input (dates, times, channels, etc), and also have some pages that will allow simple commands to be sent to the uCs (which are connected to the server as virtual com ports.)

So I'm a bit overwhelmed by the number of web development frameworks and solutions.  I've done simple stuff before just using a LAMP server and some hand coded html, css and php, but I'm not sure what the best tool set is to use for this task.

One important consideration, is if possible I want to keep the process that interacts with the uCs and logs the data to files separate from the web server.  I'm guessing I could just do communications between them using sockets?

So any thoughts on what's best to learn to do this?

[kxenos]

Well, you can't go wrong with php since you know it. If you want a little bit fancier thing you could go with AJAX which is quite easy and straight-forward. ASP is also dead-easy to learn.

[iceisfun]

I'm mainly a IIS and c# / asp.net person and would also recommend PHP, its the quickest way to hack something together if you don't already have a hosting provider and/or know c#.

[quantumvolt]

I know absolutely nothing about this, but I am very interested.

My suggestion / wish is that somebody opens a thread on this. I think many would follow a thread where you start at scratch and via a website, a PIC or an Arduino, and a sim card (and whatever more is needed) could place an instrument or voltage ref on the web. 

[Buendía]

If you want to stick to python, you may check bottle.py. It is simple (single file), far from the complexity of other frameworks, but it includes routing, template engine and a simple http server. It can also work with other python servers or with apache. You could get the user form data, generate the plot calling gnuplot and return a page with the image file.

http://bottlepy.org/docs/dev/

[Harvs]

You could get the user form data, generate the plot calling gnuplot and return a page with the image file.

That's exactly what I want to do.  I'm already using python to dynamically configure gnuplot to generate png files for reports.  I just want to do this from a web interface for other people to be able to view the results (it'll probably be used for teaching.)  I would rather use python if possible, while I know the basics of hacking together a php page (it's not hard really), the other server size software I've currently built is in python so that would be one less language that I'd have to keep in my head while completing this project.

I'll have a play around with bottle and see how far I get.

I know absolutely nothing about this, but I am very interested.

My suggestion / wish is that somebody opens a thread on this. I think many would follow a thread where you start at scratch and via a website, a PIC or an Arduino, and a sim card (and whatever more is needed) could place an instrument or voltage ref on the web. 

While I'm normally very open about sharing everything I do.  This one I can't.

[Buendía]

You could get the user form data, generate the plot calling gnuplot and return a page with the image file.

That's exactly what I want to do.  I'm already using python to dynamically configure gnuplot to generate png files for reports.  I just want to do this from a web interface for other people to be able to view the results (it'll probably be used for teaching.)  I would rather use python if possible, while I know the basics of hacking together a php page (it's not hard really), the other server size software I've currently built is in python so that would be one less language that I'd have to keep in my head while completing this project.

I'll have a play around with bottle and see how far I get.

The bottle.py tutorial covers what you need, apart from generating the plot itself. Since the default http server offered by bottle.py is single-threaded, the server will be busy handling the request (calling gnu plot) and no other requests will be served meanwhile. This is a problem if there are more than one user accessing the website at the same time. If that is required you may switch to other servers like "paste". It is a matter of installing it (a python package), and changing a line in your code. In general, it is better to have multithreaded servers since even accessing a single web page usually involves several requests (html, css, images, js files, etc).

http://bottlepy.org/docs/dev/deployment.html

A python http server may be enough if the server is going to be hosted in a machine in a LAN. If you intend to use an internet hosting service, you'll probably need to go the apache route because you may not have permissions to start your own http server. For example, I've run bottle.py apps in dreamhost shared hosting through apache. Another option, in case you have a VPS or a dedicated server, is to just run bottle.py there using a python http server like paste, and not install apache. I have done that in a minimal 128mb VPS.

Keep in mind that in case the server is multithreaded you may have more than one request at the same time, so you need to be able to generate plots in parallel, or use locks to serialize the process.

[amyk]

I'd go with Apache + PHP, if for none other than the fact that it enables you to write a webpage, then embed the dynamic code in it. As far as I know Python has no "passthrough" mode so you'll need to explicitly output the static pieces too, which can get a bit annoying.

[nctnico]

IMHO the first decission you have to make is whether you do the heavy lifting on the client or the server. The trend is to do more and more on the client with frameworks like Jquery. It allows to make your website look very nice but the price is speed. A lot of webpages work very slow nowadays.

[ivaylo]

If you are already doing Python give web2py a shot. Pretty straightforward, the book is great, as is the google group. I have 6-7 installations at work collecting data, monitoring traffic, some of them pretty involved, rarely had to write more than a few hundred lines of Python with web2py. Comes with jQuery and all the latest front end goodies so you don't have to spend time picking those.

[alxnik]

Apache + [my/postgre]SQL + php/python/whatever + ...
http://www.highcharts.com/

'nuff said. It's free for non commercial use, very very pretty and you can have it set up quite fast. You fill in the surrounding areas with php or static html (as well as the interface to the db) and you are done.

I have already implemented this for a website that collects and analyses data from solar panel installations, check the attached picture for a sample.

By far the fastest solution was to use a "cloud" based solution. I have experimented with cosm.com in which you uploaded your data as json and everything else was ready for you. Unfortunately they seem to have been sold to some other company which closed up to commmercial solutions only.

If you are interested you can check out https://github.com/alxnik/fpd which is the monitoring program I developed to input data to the db. There is no way it will work on a uC as it designed for linux, but you might get some ideas.