Community
Participate
Working Groups
Main changes: 1. Recent unicode support 2. Less memory usage Full changelog: http://www.cse.unsw.edu.au/~kleing/jflex/changelog.html I want also drop custom skeleton. It internally collect old buffer to prevent changes during read. Unfortunately this produce 2 new string for each token (one new one concat) in file. It's provoke a lot completely unnecessary GC's. If this is still important, I think will better to copy file buffer before parsing.
Thierry? What you think?
(In reply to Dawid Pakula from comment #1) > Thierry? What you think? Hi Dawid, how are you? :) Sorry, I'm quite busy since 2 months, it will be better in May, I'm planning one or 2 fixes for PDT ;) I was also looking at the recent changes in JFlex, they made really interesting changes to lower memory usage (like you said). But on my side, I would wait for version 1.8.2, it should be released soon I think, they have some pending fixes I would like to see merged before upgrading JFlex. I'm agree to drop custom skeletons, if possible. I'll have a look next month if it's ok for you. Thierry.
(In reply to Thierry BLIND from comment #2) > (In reply to Dawid Pakula from comment #1) > > Thierry? What you think? > > Hi Dawid, how are you? :) Good, I'm extremely busy due COVID-19, but I greatly reduced UI pauses and improve speed (you need latest DLTK and PDT). > Sorry, I'm quite busy since 2 months, it will be better in May, I'm planning > one or 2 fixes for PDT ;) > I was also looking at the recent changes in JFlex, they made really > interesting changes to lower memory usage (like you said). > But on my side, I would wait for version 1.8.2, it should be released soon I > think, they have some pending fixes I would like to see merged before > upgrading JFlex. > > I'm agree to drop custom skeletons, if possible. I'll have a look next month > if it's ok for you. I was sure that next version will be 1.9.0 (https://github.com/jflex-de/jflex/milestone/17) we can wait off course. Issue 153 looks promising.
New Gerrit change created: https://git.eclipse.org/r/162703
Hi, I will split the task into separate patches. The first one will simply update code source to reflect changes introduced in JFlex 1.8.2, later we can simplify our code to totally remove custom skeletons. The main change (for us) is that yychar (the number of read characters) is now of type long (previously it was of type int), see https://github.com/jflex-de/jflex/pull/605 On our side, we cannot easily propagate this change because classes like org.eclipse.wst.sse.core.internal.parser.ForeignRegion, org.eclipse.wst.xml.core.internal.parser.ContextRegionContainer, org.eclipse.wst.sse.core.internal.provisional.text.ITextRegionContainer or org.eclipse.wst.xml.core.internal.parser.regions.XMLParserRegionFactory manipulate and require offsets and region lengths to be of type int. For now we simply cast yychar as an int, anyway we never really supported 2GB php files :)
Gerrit change https://git.eclipse.org/r/162703 was merged to [master]. Commit: http://git.eclipse.org/c/pdt/org.eclipse.pdt.git/commit/?id=b16eda040544dbf85ad6512867240c15679ec2a3
Note also that I had to comment line '@SuppressWarnings({"unused", "nls"})' in our lexer definition files because it now conflicts with line '@SuppressWarnings("FallThrough")' that is automatically added by JFlex to the generated java files. I've opened an issue in JFlex to ask if it would be possible to add our own annotations again in future JFlex releases: https://github.com/jflex-de/jflex/issues/762
