Bug 474115 - Big arrays (40k lines) freezes eclipse
Summary: Big arrays (40k lines) freezes eclipse
Status: REOPENED
Alias: None
Product: z_Archived
Classification: Eclipse Foundation
Component: PDT (show other bugs)
Version: unspecified   Edit
Hardware: PC Windows 7
: P3 major (vote)
Target Milestone: ---   Edit
Assignee: PHP Core CLA
QA Contact:
URL:
Whiteboard:
Keywords: performance
Depends on: 562286
Blocks:
  Show dependency tree
 
Reported: 2015-08-03 03:45 EDT by Laurent Lyaudet CLA
Modified: 2020-05-14 10:17 EDT (History)
5 users (show)

See Also:


Attachments
Error popup (27.50 KB, image/png)
2015-08-03 03:45 EDT, Laurent Lyaudet CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Laurent Lyaudet CLA 2015-08-03 03:45:27 EDT
Created attachment 255583 [details]
Error popup

Hi,

I have a big update script in my project.
It is impossible to edit it with Eclipse.

Steps to reproduce :
1) create a PHP file using one of the two scripts given below.
2) add the generated file to your project
3) open the PHP file in eclipse (freezes ~5mins)
4) try to edit it (freezes ~5mins and an error popup appears - see attached file)

Maybe there is some buffer or hashmap which size is fixed to some small/medium value and doesn't depend of size of parsed file.
Let me know if I can help further.

Best regards,
    Laurent Lyaudet

first script :
<?php

$sTexte = "<?php\n\$array = array();\n";
for($i = 0; $i < 40000; ++$i){
  $sTexte .= "\$array []= \"It's a big string you know. Of course I know.\";\n";
}
$sTexte .= '?>';

file_put_contents('testArray.php', $sTexte);
?>

second script :
<?php

$sTexte = "<?php\n\$array = array(\n";
for($i = 0; $i < 40000; ++$i){
  $sTexte .= "  \"It's a big string you know. Of course I know.\",\n";
}
$sTexte .= ");\n?>";

file_put_contents('testArray2.php', $sTexte);
?>
Comment 1 Michal Niewrzal CLA 2015-08-04 12:12:33 EDT
It looks more like DLTK issue. Problem is around hashmap inside org.eclipse.dltk.internal.core.ModelCache.
Comment 2 Dawid Pakula CLA 2015-08-04 12:18:47 EDT
(In reply to Michal Niewrzal from comment #1)
> It looks more like DLTK issue. Problem is around hashmap inside
> org.eclipse.dltk.internal.core.ModelCache.

Yes and no. We generate 40k local fields here.
Comment 3 Michal Niewrzal CLA 2015-08-04 12:26:55 EDT
True, I will look at it.
Comment 4 Michal Niewrzal CLA 2015-08-05 10:03:03 EDT
For this case we can exclude ArrayVariableReference from Assignment processing in PHPSourceElementRequestor. This information is not used anywhere, at least I think so :) 

With this we still need to work with editor stability because from time to time PDT is freezing. It is not related to this issue, but more to file size (2MB+). One file like this can consume huge amounts of memory.
Comment 5 Eclipse Genie CLA 2015-08-06 16:15:52 EDT
New Gerrit change created: https://git.eclipse.org/r/53365
Comment 7 Michal Niewrzal CLA 2015-08-07 08:09:33 EDT
I think I fixed main problem with freezes and now it should be usable. If you will face something else related to performance just reopen this issue.
Comment 8 Sylvia Tancheva CLA 2016-06-10 10:10:37 EDT
Looks OK now, closing
Comment 9 Thierry BLIND CLA 2016-07-12 09:59:26 EDT
I'll reopen this one, see my comment for bug 298943:

"Sadly I still see UI freezes when you take sample file testArray.php from bug report 474115. Just click on "$array" and wait on highlighter to highlight all "$array" occurrences, UI will freeze for several minutes."
Comment 10 Thierry BLIND CLA 2016-07-17 18:29:11 EDT
Some additional infos: problems about UI freezes with sample file testArray.php lies in method PHPStructuredEditor#OccurrencesFinderJob.run(IProgressMonitor progressMonitor) when calling following code:
((IAnnotationModelExtension) annotationModel).replaceAnnotations(fOccurrenceAnnotations, annotationMap);

If I take code (and adapt it) from JDT method JavaEditor#OccurrencesFinderJob.run(IProgressMonitor progressMonitor), it's even worse. So it's maybe a general problem in eclipse. Any good solution is welcome ;)
Comment 11 Dawid Pakula CLA 2020-04-19 16:02:56 EDT
I collected couple issues:

1. Additional freeze on window focus - already fixed in HEAD
2. Problem with replace annotations - DLTK AnnotationModel is extremely slow due ReverseMap implementation, same problem have JDT and has been informed, will see what they do
3. OverviewRuller collect annotation for painting without synchronisation, due this for each one (warning, occurrence etc...) wait for lock. If during waiting DLTK perform long replaceAnnotations with lock, UI have to wait...
Comment 12 Eclipse Genie CLA 2020-05-01 08:54:56 EDT
New Gerrit change created: https://git.eclipse.org/r/161863
Comment 13 Eclipse Genie CLA 2020-05-01 12:17:35 EDT
New Gerrit change created: https://git.eclipse.org/r/161871