29 Oct
2014
29 Oct
'14
11:32 p.m.
Le 2014-10-28 17:24, Hugues Peccatte a écrit :
2014-10-28 20:58 GMT+01:00 Hugues Peccatte <hugues.peccatte@gmail.com mailto:hugues.peccatte@gmail.com>:
2014-10-19 21:53 GMT+02:00 Marc Delisle <marc@infomarc.info <mailto:marc@infomarc.info>>: Le 2014-10-19 15:01, Hugues Peccatte a écrit : > 2014-10-19 13:21 GMT+02:00 Marc Delisle <marc@infomarc.info <mailto:marc@infomarc.info> > <mailto:marc@infomarc.info <mailto:marc@infomarc.info>>>: > > Le 2014-10-18 14:48, Hugues Peccatte a écrit : > > 2014-10-13 12:21 GMT+02:00 Hugues Peccatte <hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com> <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com>> > > <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com> > <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com>>>>: > > > > 2014-10-12 21:06 GMT+02:00 Marc Delisle <marc@infomarc.info <mailto:marc@infomarc.info> <mailto:marc@infomarc.info <mailto:marc@infomarc.info>> > > <mailto:marc@infomarc.info <mailto:marc@infomarc.info> <mailto:marc@infomarc.info <mailto:marc@infomarc.info>>>>: > > > > Le 2014-10-12 12:57, Hugues Peccatte a écrit : > > > 2014-10-05 20:36 GMT+02:00 Hugues Peccatte <hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com> <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com>> > <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com> <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com>>> > > > <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com> > <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com>> > > <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com> <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com>>>>>: > > > > > > 2014-10-04 9:01 GMT+02:00 Hugues Peccatte <hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com> <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com>> > <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com> <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com>>> > > > <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com> > <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com>> > > <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com> <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com>>>>>: > > > > > > Le 4 oct. 2014 03:22, "Madhura Jayaratne" <madhura.cj@gmail.com <mailto:madhura.cj@gmail.com> <mailto:madhura.cj@gmail.com <mailto:madhura.cj@gmail.com>> > <mailto:madhura.cj@gmail.com <mailto:madhura.cj@gmail.com> <mailto:madhura.cj@gmail.com <mailto:madhura.cj@gmail.com>>> > > > <mailto:madhura.cj@gmail.com <mailto:madhura.cj@gmail.com> > <mailto:madhura.cj@gmail.com <mailto:madhura.cj@gmail.com>> > > <mailto:madhura.cj@gmail.com <mailto:madhura.cj@gmail.com> <mailto:madhura.cj@gmail.com <mailto:madhura.cj@gmail.com>>>>> a écrit : > > > > > > > > > > > > > > > > > > > > > > On Sat, Oct 4, 2014 at 1:24 AM, Hugues Peccatte > > > <hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com> <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com>> > > <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com> <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com>>> > > <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com> > <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com>> > > <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com> <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com>>>>> > > > wrote: > > > >> > > > >> 2014-10-03 12:26 GMT+02:00 Marc Delisle <marc@infomarc.info <mailto:marc@infomarc.info> <mailto:marc@infomarc.info <mailto:marc@infomarc.info>> > <mailto:marc@infomarc.info <mailto:marc@infomarc.info> <mailto:marc@infomarc.info <mailto:marc@infomarc.info>>> > > > <mailto:marc@infomarc.info <mailto:marc@infomarc.info> > <mailto:marc@infomarc.info <mailto:marc@infomarc.info>> <mailto:marc@infomarc.info <mailto:marc@infomarc.info> > <mailto:marc@infomarc.info <mailto:marc@infomarc.info>>>>>: > > > >>> > > > >>> Hi Hugues, > > > >>> I retested this morning on a laptop, importing a > > SQL file > > > containing > > > >>> 10000 employees from the sample employees > > database. This is > > > a small file > > > >>> (660 KB). > > > >>> > > > >>> Current master: 3 min 25 sec (and ends with > > JSON.parse: > > > unexpected > > > >>> character) > > > >>> > > > >>> Current Tithugues/stringFunctions_master: 2 > min 10 > > sec (same > > > js error) > > > >>> > > > >>> Current QA_4_2: 0 min 5 sec > > > >>> > > > >>> There has been improvement, but we cannot > release > > 4.3 with > > > this import > > > >>> speed. > > > >>> > > > >>> -- > > > >>> Marc Delisle | phpMyAdmin > > > >> > > > >> > > > >> Hi, > > > >> > > > >> I agree… But I'm afraid this is linked to > > multibytes functions… > > > >> Maybe we shouldn't use the multibytes functions > > everywhere… > > > >> > > > >> I'll still try to improve performances. > > > >> > > > >> Hugues. > > > >> > > > > > > > > Indeed, I also think that we should use mb_* > > function only > > > when necessary and choice to use them should be made > > in case by > > > case basis. > > > > > > > > -- > > > > Thanks and Regards, > > > > > > > > Madhura Jayaratne > > > > > > Hi, > > > > > > I didn't push my commits, but that's what I've > started. I > > > replaced the mb_* calls by standard calls on > configuration > > > variables, reserved words, etc. > > > > > > Hugues. > > > > > > > > > Hi, > > > > > > Out of desperation, I try another algorithm. Instead of > > buffering > > > data until SQL delimiter, I'll try to parse all lines. > > > So, I won't parse 1000 times a buffer of 50000 > characters, > > but less > > > than 10 times many buffers of 500 characters. I hope > this > > will be > > > faster. > > > > > > Hugues. > > > > > > > > > Hi, > > > > > > The new algorithm is over. There are still some controls to > > add, but it > > > is usable with the file in this ticket: [1] > > > You can find my modifications here: [2] > > > > > > Marc, is it faster for you ? > > > It seems that I won ~33% of time. We're still far from 5 > seconds… > > > Maybe I'll try to use standard PHP functions to see the > > difference. If > > > the standard PHP functions are really faster, I'll try > to add > > an option > > > to use mb_* functions or standard PHP functions, as you > said. > > > > > > [1] https://sourceforge.net/p/phpmyadmin/bugs/4536/ > > > [2] > > > > > > https://github.com/Tithugues/phpmyadmin/tree/stringFunctions_useStandardFunctions_master > > > > Hi Hugues, > > yes it's faster. With the same testing conditions, the import > > takes 1 > > min 20 sec. > > > > > > -- > > Marc Delisle | phpMyAdmin > > > > > > Thanks for your feedback. > > I'll try another another improvement to be faster. > > > > Note for my self: > > * read X characters but don't restart the search from 0 each time > > * search for the escaped quote with a lookbehind expression, > > something like `(?!<\\)(\\\\)*'` > > > > Hugues. > > > > > > Hi, > > > > As asked by Marc, I added on option to import by reading as a > multibytes > > string or not. > > The default configuration won't read as multibytes string (because > it's > > too long…). It seems that the DnD to import doesn't use the default > > configuration, so what ever you define as default, it won't be use in > > this process. > > Should we create a ticket for this ? I think it's possible to get > it in > > javascript. > > > > Hugues. > > > > With the new option "Read as multibytes" left to its default disabled > value, my test file imports in 5 seconds. > > When this option is ticked, it takes 60 seconds. > > Note that in both cases, I get a Javascript error at the end (the error > report does not show the same error in Firefox than in Chromium). Can > anyone confirm this? > > > -- > Marc Delisle | phpMyAdmin > > > Marc, > > Could we please have both errors? I don't have it on Chrome. I'll try on > Firefox and IE. > > Thanks, > Hugues. >From Iceweasel 24.8.1: { "pma_version": "4.3.0-dev", "browser_name": "FIREFOX", "browser_version": "24.0", "user_os": "Linux", "server_software": "Apache/2.2.22 (Debian)", "user_agent_string": "Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20140924 Firefox/24.0 Iceweasel/24.8.1", "locale": "en", "configuration_storage": "enabled", "php_version": "5.4.4-14+deb7u14", "exception_type": "js", "exception": { "mode": "stack", "name": "SyntaxError", "message": "JSON.parse: unexpected character", "stack": [ { "func": ".parseJSON", "args": "", "line": 2, "column": "", "context": [ "/*! jQuery v1.8.3 jquery.com <http://jquery.com> | jquery.org/license <http://jquery.org/license> */\r", "(function(e,t){function _(e){var t=M[e]={};return v.each(e.split(y),functio//...", "", "function sprintf() {", "/*", " * Copyright (c) 2013 Kevin van Zonneveld (http://kvz.io)", " * and Contributors (http://phpjs.org/authors)" ], "filename": "jquery/jquery-1.8.3.min.js" }, { "func": "PMA_console.initialize/<", "args": "", "line": 164, "column": "", "context": [ " $('#pma_console_options input[name=current_query]').prop('c//...", " PMA_console.updateConfig();", " });", "", " $(document).ajaxComplete(function (event, xhr) {", " var data = $.parseJSON(xhr.responseText);", " PMA_console.ajaxCallback(data);", " });", "", " PMA_console.isInitialized = true;", " }" ], "filename": "console.js" }, { "func": "ErrorReport.wrap_function/new_func", "args": "", "line": 262, "column": "", "context": [ " */", " wrap_function: function (func) {", " if (!func.wrapped) {", " var new_func = function () {", " try {", " return func.apply(this, arguments);", " } catch (x) {", " TraceKit.report(x);", " }", " };", " new_func.wrapped = true;" ], "filename": "error_report.js" }, { "func": "v.event.dispatch", "args": "", "line": 2, "column": "", "context": [ "/*! jQuery v1.8.3 jquery.com <http://jquery.com> | jquery.org/license <http://jquery.org/license> */\r", "(function(e,t){function _(e){var t=M[e]={};return v.each(e.split(y),functio//...", "", "function sprintf() {", "/*", " * Copyright (c) 2013 Kevin van Zonneveld (http://kvz.io)", " * and Contributors (http://phpjs.org/authors)" ], "filename": "jquery/jquery-1.8.3.min.js" }, { "func": "$event.dispatch", "args": "", "line": 372, "column": "", "context": [ "$event.dispatch = function( event ){", "\tif ( $.data( this, \"suppress.\"+ event.type ) - new Date().getTime() > 0 ){", "\t\t$.removeData( this, \"suppress.\"+ event.type );", "\t\treturn;", "\t}", "\treturn $dispatch.apply( this, arguments );", "};", "", "// event fix hooks for touch events...", "var touchHooks = ", "$event.fixHooks.touchstart = " ], "filename": "jquery/jquery.event.drag-2.2.js" }, { "func": "v.event.add/u", "args": "", "line": 2, "column": "", "context": [ "/*! jQuery v1.8.3 jquery.com <http://jquery.com> | jquery.org/license <http://jquery.org/license> */\r", "(function(e,t){function _(e){var t=M[e]={};return v.each(e.split(y),functio//...", "", "function sprintf() {", "/*", " * Copyright (c) 2013 Kevin van Zonneveld (http://kvz.io)", " * and Contributors (http://phpjs.org/authors)" ], "filename": "jquery/jquery-1.8.3.min.js" }, { "func": "v.event.trigger", "args": "", "line": 2, "column": "", "context": [ "/*! jQuery v1.8.3 jquery.com <http://jquery.com> | jquery.org/license <http://jquery.org/license> */\r", "(function(e,t){function _(e){var t=M[e]={};return v.each(e.split(y),functio//...", "", "function sprintf() {", "/*", " * Copyright (c) 2013 Kevin van Zonneveld (http://kvz.io)", " * and Contributors (http://phpjs.org/authors)" ], "filename": "jquery/jquery-1.8.3.min.js" }, { "func": "v.event.trigger", "args": "", "line": 2, "column": "", "context": [ "/*! jQuery v1.8.3 jquery.com <http://jquery.com> | jquery.org/license <http://jquery.org/license> */\r", "(function(e,t){function _(e){var t=M[e]={};return v.each(e.split(y),functio//...", "", "function sprintf() {", "/*", " * Copyright (c) 2013 Kevin van Zonneveld (http://kvz.io)", " * and Contributors (http://phpjs.org/authors)" ], "filename": "jquery/jquery-1.8.3.min.js" }, { "func": "T", "args": "", "line": 2, "column": "", "context": [ "/*! jQuery v1.8.3 jquery.com <http://jquery.com> | jquery.org/license <http://jquery.org/license> */\r", "(function(e,t){function _(e){var t=M[e]={};return v.each(e.split(y),functio//...", "", "function sprintf() {", "/*", " * Copyright (c) 2013 Kevin van Zonneveld (http://kvz.io)", " * and Contributors (http://phpjs.org/authors)" ], "filename": "jquery/jquery-1.8.3.min.js" }, { "func": ".send/r", "args": "", "line": 2, "column": "", "context": [ "/*! jQuery v1.8.3 jquery.com <http://jquery.com> | jquery.org/license <http://jquery.org/license> */\r", "(function(e,t){function _(e){var t=M[e]={};return v.each(e.split(y),functio//...", "", "function sprintf() {", "/*", " * Copyright (c) 2013 Kevin van Zonneveld (http://kvz.io)", " * and Contributors (http://phpjs.org/authors)" ], "filename": "jquery/jquery-1.8.3.min.js" } ], "useragent": "Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20140924 Firefox/24.0 Iceweasel/24.8.1", "incomplete": "false", "uri": "db_import.php?target=" }, "script_name": "db_import.php", "microhistory": { "pages": [ { "hash": "index.php?db=&table=&server=1&target=&token=c455fd8a0595a5032d18c07a64e7ab0a" }, { "hash": "db_structure.php?db=test&table=&server=1&target=&token=c455fd8a0595a5032d18c07a64e7ab0a", "params": { "opendb_url": "db_structure.php", "safari_browser": "0", "collation_connection": "utf8mb4_general_ci", "lang": "en", "server": "1", "text_dir": "ltr", "pma_text_default_tab": "Browse", "pma_text_left_default_tab": "Structure", "LimitChars": "50", "pftext": "", "confirm": "true", "LoginCookieValidity": "144000", "logged_in": "true" } }, { "hash": "index.php?db=&table=&server=1&target=&token=c455fd8a0595a5032d18c07a64e7ab0a", "params": { "opendb_url": "db_structure.php", "safari_browser": "0", "collation_connection": "utf8mb4_general_ci", "lang": "en", "server": "1", "text_dir": "ltr", "pma_text_default_tab": "Browse", "pma_text_left_default_tab": "Structure", "LimitChars": "50", "pftext": "", "confirm": "true", "LoginCookieValidity": "144000", "logged_in": "true" } }, { "hash": "db_structure.php?db=test&table=&server=1&target=&token=c455fd8a0595a5032d18c07a64e7ab0a", "params": { "opendb_url": "db_structure.php", "safari_browser": "0", "collation_connection": "utf8mb4_general_ci", "lang": "en", "server": "1", "text_dir": "ltr", "pma_text_default_tab": "Browse", "pma_text_left_default_tab": "Structure", "LimitChars": "50", "pftext": "", "confirm": "true", "LoginCookieValidity": "144000", "logged_in": "true" } }, { "hash": "db_import.php?db=test&table=&server=1&target=&token=c455fd8a0595a5032d18c07a64e7ab0a", "params": { "opendb_url": "db_structure.php", "safari_browser": "0", "collation_connection": "utf8mb4_general_ci", "lang": "en", "server": "1", "text_dir": "ltr", "pma_text_default_tab": "Browse", "pma_text_left_default_tab": "Structure", "LimitChars": "50", "pftext": "", "confirm": "true", "LoginCookieValidity": "144000", "logged_in": "true" } } ], "current_index": "5" } } -- Marc Delisle | phpMyAdmin Hi, It seems that the new algorithm is over, see [1]. The unit tests are now ok (except for the one failing on master). I did some tests with escaped quotes (with quote or backslash), delimiters into strings, modified delimiters. Marc, do you still have your error please? (I didn't work on it, but I hope that your issue is linked to the not-finished version…)
On my home machine, sadly I still have the js error after importing. Tested on Iceweasel 31.2.
Surely someone else can reproduce this problem on Debian wheezy? Because if it's only me, I don't care too much ;)
With 3b3017d7, my test file (10000 rows) imports in 40 seconds in multibyte mode.
Feel free to look at the new algorithm and to tell me if you see any flaw. If you can lead to an error with a correct SQL file, please send it to me. Thanks for your feedback. [1] https://github.com/Tithugues/phpmyadmin/tree/stringFunctions_useStandardFunctions_master Hugues.Hi,
Since my previous email, I did some modifications to manage the display of queries in a different way (closer to the current display).
But there is something strange with the attachment… I can import the file (without error and all content executed), but on master, if I paste the content of the file into the SQL console, I've an error… Is it the expected behavior please? I don't see any exotic content in this file… On my branch, I don't have an error, but only the first statement (DROP PROCEDURE IF EXISTS) is executed…
Thanks, Hugues.
--
Marc Delisle | phpMyAdmin