<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">2014-10-13 12:21 GMT+02:00 Hugues Peccatte <span dir="ltr"><<a href="mailto:hugues.peccatte@gmail.com" target="_blank">hugues.peccatte@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div class="h5">2014-10-12 21:06 GMT+02:00 Marc Delisle <span dir="ltr"><<a href="mailto:marc@infomarc.info" target="_blank">marc@infomarc.info</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Le 2014-10-12 12:57, Hugues Peccatte a écrit :<br>
<span>> 2014-10-05 20:36 GMT+02:00 Hugues Peccatte <<a href="mailto:hugues.peccatte@gmail.com" target="_blank">hugues.peccatte@gmail.com</a><br>
</span>> <mailto:<a href="mailto:hugues.peccatte@gmail.com" target="_blank">hugues.peccatte@gmail.com</a>>>:<br>
<span>><br>
> 2014-10-04 9:01 GMT+02:00 Hugues Peccatte <<a href="mailto:hugues.peccatte@gmail.com" target="_blank">hugues.peccatte@gmail.com</a><br>
</span>> <mailto:<a href="mailto:hugues.peccatte@gmail.com" target="_blank">hugues.peccatte@gmail.com</a>>>:<br>
<span>><br>
> Le 4 oct. 2014 03:22, "Madhura Jayaratne" <<a href="mailto:madhura.cj@gmail.com" target="_blank">madhura.cj@gmail.com</a><br>
</span>> <mailto:<a href="mailto:madhura.cj@gmail.com" target="_blank">madhura.cj@gmail.com</a>>> a écrit :<br>
<span>><br>
><br>
> ><br>
> ><br>
> ><br>
> > On Sat, Oct 4, 2014 at 1:24 AM, Hugues Peccatte<br>
</span>> <<a href="mailto:hugues.peccatte@gmail.com" target="_blank">hugues.peccatte@gmail.com</a> <mailto:<a href="mailto:hugues.peccatte@gmail.com" target="_blank">hugues.peccatte@gmail.com</a>>><br>
<span>> wrote:<br>
> >><br>
> >> 2014-10-03 12:26 GMT+02:00 Marc Delisle <<a href="mailto:marc@infomarc.info" target="_blank">marc@infomarc.info</a><br>
</span>> <mailto:<a href="mailto:marc@infomarc.info" target="_blank">marc@infomarc.info</a>>>:<br>
<div><div>> >>><br>
> >>> Hi Hugues,<br>
> >>> I retested this morning on a laptop, importing a SQL file<br>
> containing<br>
> >>> 10000 employees from the sample employees database. This is<br>
> a small file<br>
> >>> (660 KB).<br>
> >>><br>
> >>> Current master: 3 min 25 sec (and ends with JSON.parse:<br>
> unexpected<br>
> >>> character)<br>
> >>><br>
> >>> Current Tithugues/stringFunctions_master: 2 min 10 sec (same<br>
> js error)<br>
> >>><br>
> >>> Current QA_4_2: 0 min 5 sec<br>
> >>><br>
> >>> There has been improvement, but we cannot release 4.3 with<br>
> this import<br>
> >>> speed.<br>
> >>><br>
> >>> --<br>
> >>> Marc Delisle | phpMyAdmin<br>
> >><br>
> >><br>
> >> Hi,<br>
> >><br>
> >> I agree… But I'm afraid this is linked to multibytes functions…<br>
> >> Maybe we shouldn't use the multibytes functions everywhere…<br>
> >><br>
> >> I'll still try to improve performances.<br>
> >><br>
> >> Hugues.<br>
> >><br>
> ><br>
> > Indeed, I also think that we should use mb_* function only<br>
> when necessary and choice to use them should be made in case by<br>
> case basis.<br>
> ><br>
> > --<br>
> > Thanks and Regards,<br>
> ><br>
> > Madhura Jayaratne<br>
><br>
> Hi,<br>
><br>
> I didn't push my commits, but that's what I've started. I<br>
> replaced the mb_* calls by standard calls on configuration<br>
> variables, reserved words, etc.<br>
><br>
> Hugues.<br>
><br>
><br>
> Hi,<br>
><br>
> Out of desperation, I try another algorithm. Instead of buffering<br>
> data until SQL delimiter, I'll try to parse all lines.<br>
> So, I won't parse 1000 times a buffer of 50000 characters, but less<br>
> than 10 times many buffers of 500 characters. I hope this will be<br>
> faster.<br>
><br>
> Hugues.<br>
><br>
><br>
> Hi,<br>
><br>
> The new algorithm is over. There are still some controls to add, but it<br>
> is usable with the file in this ticket: [1]<br>
> You can find my modifications here: [2]<br>
><br>
> Marc, is it faster for you ?<br>
> It seems that I won ~33% of time. We're still far from 5 seconds…<br>
> Maybe I'll try to use standard PHP functions to see the difference. If<br>
> the standard PHP functions are really faster, I'll try to add an option<br>
> to use mb_* functions or standard PHP functions, as you said.<br>
><br>
> [1] <a href="https://sourceforge.net/p/phpmyadmin/bugs/4536/" target="_blank">https://sourceforge.net/p/phpmyadmin/bugs/4536/</a><br>
> [2]<br>
> <a href="https://github.com/Tithugues/phpmyadmin/tree/stringFunctions_useStandardFunctions_master" target="_blank">https://github.com/Tithugues/phpmyadmin/tree/stringFunctions_useStandardFunctions_master</a><br>
<br>
</div></div>Hi Hugues,<br>
yes it's faster. With the same testing conditions, the import takes 1<br>
min 20 sec. </blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<span><br>
--<br>
Marc Delisle | phpMyAdmin<br></span></blockquote><div><br></div></div></div><div>Thanks for your feedback.</div><div>I'll try another another improvement to be faster.</div><div><br></div><div>Note for my self:</div><div>* read X characters but don't restart the search from 0 each time</div><div>* search for the escaped quote with a lookbehind expression, something like `(?!<\\)(\\\\)*'`</div><span class="HOEnZb"><font color="#888888"><div><br></div><div>Hugues.</div></font></span></div></div></div>
</blockquote></div><br></div><div class="gmail_extra">Hi,</div><div class="gmail_extra"><br></div><div class="gmail_extra">As asked by Marc, I added on option to import by reading as a multibytes string or not. </div><div class="gmail_extra">The default configuration won't read as multibytes string (because it's too long…). It seems that the DnD to import doesn't use the default configuration, so what ever you define as default, it won't be use in this process.</div><div class="gmail_extra">Should we create a ticket for this ? I think it's possible to get it in javascript.</div><div class="gmail_extra"><br></div><div class="gmail_extra">Hugues. </div></div>