[Phpmyadmin-devel] GSoC Removal of Frames : New Navigation

Wed Jul 4 00:58:35 CEST 2012

On 03/07/12 21:07, Dieter Adriaenssens wrote:
> 2012/7/3 Rouslan Placella<rouslan at placella.com>:
>> On 02/07/12 21:28, Dieter Adriaenssens wrote:
>>> Hi Rouslan,
>>>
>>> I was just reading your weekly report. I have a question about the
>>> 'refreshing of navigation' :
>>> You mention that the state of the navigation (which branches are open)
>>> has to be remembered, when sending it over to the client/browser. Does
>>
>> I think that I just worded it badly. The state of the tree is never, in
>> fact, recorded anywhere. Instead it's calculated with a JS function.
>>
>>> this mean that the entire navigation tree is generated server side on
>>> every change to the nav tree (f.e. clicking on a branch to open it, or
>>
>> When clicking on a branch to open, only that branch, its immediate
>> children and all its parents are loaded into memory. Then only its
>> immediate children are rendered into HTML.
>>
>>> when tables/databases are added/deleted/renamed/...)? Or only in case
>>> of a full refresh (or first load)?
>>
>> On first load, only the list of databases (limited by $cfg['
>> MaxDbList']) is loaded into memory and rendered.
>>
>> A full refresh is used for when a db/table is added, removed, renamed,
>> copied, etc. During the full refresh, only the nodes that were visible
>> before the refresh are loaded into memory and rendered. The only
>> expection to this are tables, as any request to render a table will
>> force their children to also be loaded and rendered. The idea here seems
>> nice as we can avoid some extra ajax requests after loading a list of
>> tables. However this proved to be slow during full refreshes as all
>> index and column information my have to be loaded for dozens of tables.
>>
>>> Would it make sense to keep the nav tree state stored client side and
>>> let the script repopulate the tree based on data it gets from the
>>> server, after a request initiated by user interaction (navigating the
>>> tree) or when the server sends new/changed/deleted tables when
>>> something is changed by the user in the main panel (triggering
>>> refreshNavigation)?
>>>
>>> Or would this overcomplicate things?
>>
>> Sorry, I'm not sure what you are trying to say here. That said, I
>> haven't really explained to anyone how the whole thing works, so maybe I
>> should do that. So here is the basic workflow as it is currently in my repo:
>>
>> During the initial page load, the navigation is not sent to the client
>> at all, but just the links at the top of the panel and the logo.
>>
>> As soon as the ready event is fired on the page, the list of databases
>> is fetched via ajax. During this request only these databases are loaded
>> into memory and rendered (but none of their children).
>>
>> Then when a user clicks on a link to expand a particular branch, the
>> path to that node (at this point it's just "root.sakila", for example)
>> is sent to the server. The server will then load the databases into
>> memory and check what kind of items are present in the db (tables,
>> views, procedures, events). If there are more than one type of items,
>> then containers for them (but not the items themselves) are loaded into
>> memory, rendered and sent back to the client. Otherwise, if there are
>> only tables present in a db, then the tables themselves are loaded into
>> memory, rendered and sent back.
>>
>> Let's say that we have tables and events in the sakila db and we opened
>> its branch. At this point the list of containers is received from the
>> server and attached to the sakila node. If we click on "Tables", the
>> following path is sent to the server "root.sakila.tables" and in the
>> reply we will receive the list of tables and for each table a list of
>> columns and a list of indexes, if any.
>>
>> This gets a bit more complicated when nodes are grouped. Whether this
>> happens depends on the names of items and the
>> $cfg['LeftFrameDBSeparator'], $cfg['LeftFrameTableSeparator '] and
>> $cfg['LeftFrameTableLevel'] directives.
>> If we have a hyphen as a db separator and sakila_one and sakila_two dbs,
>> then when we try to expand the tables branch in sakila_one, the
>> following paths will be sent to the server for processing: actual_path
>> ->  "root.sakila_two.tables"; virtual_path ->  "root.sakila_.two.Tables".
>>
>> As far as refreshing goes, there is a JS function that traverses the
>> rendered tree on the page and finds the deepest nodes in the hierarchy
>> that have a "minus" icon near them. Then the list paths to these node is
>> compiled and sent to the server. The server then knows how deep to load
>> each branch into memory and which parts should be rendered. Let's look
>> at a concrete example. Here's a tree that we have started browsing:
>>
>> + phpmyadmin
>> - sakila
>>       - film
>>           + columns
>>       + film_actor
>>       - country
>>           + columns
>>           - indexes
>>               someindex
>>               someindex2
>> - testdb
>>       + tables
>>       - events
>>           someevent
>>           someevent2
>> + mysql
>> + information_schema
>>
>> If we hit refresh. Then the following paths will be sent to the server:
>>
>> root.sakila.tables.country.indexes
>> root.sakila.tables.film
>> root.testdb.events
>>
>> The server will load into memory, all tables, columns and indexes from
>> sakila and render all of them. It will check if there are still tables
>> in testdb, but it won't load any of them. And it will load and render
>> the events from testdb. The resulting HTML code is then sent back to the
>> server and the tree that is on the page is simply swapped for the new one.
>
> Hi Rouslan,
>
> Thanks for explaining how the navigation is generated. It is clearer to me now.
>
> So when clicking on a table, an AJAX request is sent to the server to
> generate HTML only for everything below (columns/indexes) that table,
> which on it's turn is added by a JS script to the HTML defining the
> current expanded tree in the navigation?
> Or is the structure of the parent nodes generated as well?

You are correct, the parent nodes need to be loaded into memory in the 
form of objects that inherit from the Node class. This way we can have a 
representation of a valid branch of the tree. Then only the active node 
and it's children are rendered into HTML (which is the most expensive 
operation).

>> The improvements that I was talking about on the blog for this system
>> would be partial refreshes and removal of preloading for children of
>> table nodes.
>>
>> * Partial refreshes would be nice when we add/modify a table, for
>> example. In this case, we only need to reload the database, not the
>> whole tree, but we need to know which database to reload, so it would be
>> necessary to pass some parameters to the refreshing function, which at
>> the moment takes none.
>
> Where would this parameter come from? I guess it would come from the
> process that renamed/removed/added a table/database/column.

That's right. And it means that it's necessary to go every piece of JS 
code that interacts with the DBMS in a way that requires a refresh of 
some part of the navigation tree.

>> * Preloading of indexes and columns for tables seemed like a good idea
>> when I was first writing the code, but I didn't foresee that it would
>> considerable slow down the refreshing of the tree, so it need to go.
>
> Can't you do a two stage refreshing? First the visible things (tables
> of a database, when opening a database with only tables (no events,
> ..)) and then in a second stage (when the tree is populated with
> tables) the level below that (in this example columns and indexes for
> every table).
> This way the users sees the tree populated quite fast, because only
> the tables are displayed. In the background, what's below the tables
> is populated, resulting in a quick opening of a node when clicking on
> a table, because what's below is already loaded.
>
> With database with many tables this might be a problem, so there
> probably should be some kind of limit to the amount of preloading.

I think that I would like to keep this simple for the moment. I might 
dedicate it more time at a later stage.

> Kind regards,
>
> Dieter
>
>> I hope this makes it clearer how things are done in the new navigation.
>> Feel free to ask if there are still things that are unclear, and any
>> suggestion are always welcome :)
>>
>> Bye,
>> Rouslan
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Phpmyadmin-devel mailing list
>> Phpmyadmin-devel at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/phpmyadmin-devel
>
>
>