[Phpmyadmin-devel] GSoC - Server Stats

[Philip Frank]

2010/4/2 Marc Delisle <marc at infomarc.info>:
> Philip Frank a écrit :
>> 2010/4/1 Marc Delisle <marc at infomarc.info>:
>>> Philip Frank a écrit :
>>>> 2010/4/1 Marc Delisle <marc at infomarc.info>:
>>>>> Philip Frank a écrit :
>>>>>> Hello everyone,
>>>>>>
>>>>>> I'm a student from Munich, Germany and stumbled across the GSoC Idea
>>>>>> of collecting the data that is shown on the server status page and
>>>>>> rendering them as graphs over time. I will apply in the next days,
>>>>>> here is what I thought of implementing when I'm accepted:
>>>>>>
>>>>>> Data collection: In the past, I used rrdtool to collect stats on a
>>>>>> server. Its datastore has a limited size - over time old entries get
>>>>>> merged together and the data gets thinner, but is never completely
>>>>>> deleted. Unfortunately, it is not very portable. Therefor, I would
>>>>>> implement a similarily behaving datastore ontop of a mysql table. I
>>>>>> think of storing each set of measures as a row and limit the number of
>>>>>> total rows, making room for new entries by combining one pair of
>>>>>> neighbouring old entries together.
2010/4/2 Raj Shekhar <rajlist at rajshekhar.net>:
> In infinite wisdom Philip Frank <ich at philipfrank.de> wrote:
>
>>
>> Data collection: In the past, I used rrdtool to collect stats on a
>> server. Its datastore has a limited size - over time old entries get
>> merged together and the data gets thinner, but is never completely
>> deleted. Unfortunately, it is not very portable.
> http://infomarc.info
2010/4/2 Raj Shekhar <rajlist at rajshekhar.net>:
> In infinite wisdom Philip Frank <ich at philipfrank.de> wrote:
>
>>
>> Data collection: In the past, I used rrdtool to collect stats on a
>> server. Its datastore has a limited size - over time old entries get
>> merged together and the data gets thinner, but is never completely
>> deleted. Unfortunately, it is not very portable.
>
> The size of the RRD file does not increase over time.  THat is a
> feature, not a bug :-) .  Merging of the older data (reducing the
> granularity) is a side effect of this.  In my opinion, the way RRD
> handles that is quite intelligent.  It would be hard to get that
> functionality right using mysql and rollup tables.
>
> Why do you say that RRDs are not portable?  I have seen them work on
> bsd/linux (and on 32/64 bit).
>
> --
> Raj Shekhar

Hi Raj,

thanks for your feedback! The only way to write or access rrds is the
rrdtool executable. You can only get data out by printing it out as
graphs, thats what I mean by not portable.
>From what Marc wrote and reading the past discussions on the list I
rethough of my project to not require any configurations on the server
besides running pma. I won't use any server side datastore so rrdtool
is off the table, anyway.

>>>>> Hi Philip,
>>>>> so you plan that an external process will collect data? Please have a
>>>>> look on this list archives, this was discussed a few days ago.
>>>>>
>>>> Hi Marc,
>>>> i thought the process could be triggered by a cron, but if that is not
>>>> an option, historic data can not be collected, i guess. How about a
>>>> live monitoring tool? I could create a visualisation of the ratios of
>>>> the different query types. Constantly updating that with ajax would be
>>>> nice, and maybe even collecting the data in the browser and plotting
>>>> graphes.
>>> As discussed on the list, asking users to setup cron is out of
>>> phpMyAdmin's scope. Live monitoring tool is interesting.
>>>
>>> --
>>> Marc Delisle
>>> http://infomarc.info
>>>
>>
>> Alright, that live monitoring could be awesome, too. The user could
>> select what values should be measured, they would be constantly
>> updated via ajax and it would be graphed with processing.js.
>> The data could be stored as long as the browser window is open, so it
>> could still be displayed on a time axis. At its best, one could just
>> leave pma running in his browser while testing different
>> configurations.
>>
>> With the html5 client storage, even saving snapshots persistently and
>> comparing them could be done.
>>
>> How does this sound?
>
> Before reading your post about "as long as the browser window is open" I
> had the same idea.
>
> About selecting what values should be measured, I think that this
> interface should at least propose a list of the most popular things to
> measure, to guide the admin. I guess that admins will want to use this
> to find problems on their servers.
>
> --
> Marc Delisle



Hi Marc,

There are really a lot of values displayed, and I don't have a clue
what most of them say. Ordering them somehow when allowing the admin
to pick is a good idea. Some show up in red on my server, I guess
these could also be more prominent since they possibly indicate a
problem.

I found this reference guide for the status variables:
http://dev.mysql.com/doc/refman/5.0/en/server-status-variables.html
Do you know any other good documents that describe what these values
are all about?


I wrote most of my application today, this is my list of milestones:
- JSON Protocol is defined
- PHP middleware running
- Client side datastore planned and implemented
- Mocked UI & created HTML
- Data gets filtered and recorded into datastore
- Datastore benchmarked, browsers don't crash
- Documentation on datastore written
- Study processing.js
- JS object hierarchy for plotting planned
- Defined interface to feed plotter from datastore
- Glue code to datastore is written
- Plotters' common features implemented
- Bar graph plotter done
- Curve plotter for temporal data done
- Documentation about plotters written
- Bonus: Implement additional plotters

I'm pretty confident this can be done in the allocated time.

>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Phpmyadmin-devel mailing list
> Phpmyadmin-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/phpmyadmin-devel
>