Community
Participate
Working Groups
When launching gdb 7.6 in Luna, I see in gdb traces that breakpoints are sometimes set and deleted again when launching: ... 039,421 23-interpreter-exec --thread-group i1 console "show endian" 039,421 ~"The target endianness is set automatically (currently little endian)\n" 039,421 23^done 039,421 (gdb) 039,424 24-break-insert -f xxx.cpp:393 039,424 25-break-insert -f yyy.h:156 039,424 26-break-insert -f zzz.cpp:654 039,710 24^done,bkpt={number="1",type="breakpoint",disp="keep",enabled="y",addr="0x000000000053fd07"\ ,func="xxx::TestBody()"\ ,file="xxx",fullname="xxx.cpp",line="393",thread-groups=["i1"],times="0",original-location\ ="xxx:393"\ } 039,710 (gdb) 039,864 27-break-delete 1 045,791 25^done,bkpt={number="2",type="breakpoint",disp="keep",enabled="y",addr="<MULTIPLE>",times="\ ... I don't see why "break-delete 1" is being called, I have not set up any filtering or anything like that would required removing the breakpoint again. This seems to happen rather randomly, where sometimes the breakpoint gets removed while launching and sometimes not. Also, requiring to stop in main seems to help somewhat, but not in all cases.
Here the output of another run where two of three breakpoints actually get deleted for seemingly no reason: ... 165,449 23-interpreter-exec --thread-group i1 console "show endian" 165,449 ~"The target endianness is set automatically (currently little endian)\n" 165,450 23^done 165,450 (gdb) 165,452 24-break-insert -f xxx:654 165,452 25-break-insert -f yyy:393 165,452 26-break-insert -f zzz:156 165,741 24^done,bkpt={number="1",type="breakpoint",disp="keep",enabled="y",addr="0x00000000005443f2"\ ,func="xxx::TestBody()\ ",file="xxx",fullname="xxx",line="654",thread-groups=["i1"],times="0",original-locatio\ n="xxx:654\ "} 165,742 (gdb) 165,743 25^done,bkpt={number="2",type="breakpoint",disp="keep",enabled="y",addr="0x000000000053fd07"\ ,func="yyy::TestBody()"\ ,file="yyy",fullname="yyy",line="393",thread-groups=["i1"],times="0",original-location\ ="yyy:393"\ } 165,743 (gdb) 165,897 27-break-delete 2 165,897 28-break-delete 1 171,685 26^done,bkpt={number="3",type="breakpoint",disp="keep",enabled="y",addr="<MULTIPLE>",times="\ 0",original-location="zzz.h:\ 156"},{number="3.1",enabled="y",addr="0x00000000005b62cd",func=" ...
Anybody have any suggestions? It's pretty hard to debug something when the execution doesn't stop at a lot of breakpoints.
I have tried to reproduce this but have not had any luck, Can you provide any more hints that can help us reproduce this problem ? e.g. There are some features like Run-to-line and Step-into-selection that need to insert a temporary break point to reach its location, would this problem happen when exercising these or any other feature ? Could you describe any specifics of your launch configuration e.g. All-stop mode / non-stop mode. It would be very useful if you could provide a small program that helps to reproduce this issue
Alvaro, I will see what I can do, but I am not hopeful here. I was not using any specific debug options, pretty much default. In fact, I was actually cycling through all the options to see if one of them would make a difference, but none did. I was also not using any run-to-line or anything like that. What happens is that even if you just bring up the task and stop in main, you can already see some breakpoints don't have the check mark set, since they have been deleted again. Can you maybe tell me what situations exist in the code the could cause Eclipse to call break-delete after it has already inserted the breakpoint? Maybe that can help me figure out how to reproduce it in a small sample program.
I can't reproduce it either, could be a race condition. The latest breakpoint related changes were for https://bugs.eclipse.org/bugs/show_bug.cgi?id=433044 and https://bugs.eclipse.org/bugs/show_bug.cgi?id=389945. The former is almost the opposite of this issue. Anyway, I'll try to reproduce it looking at the code. Marc will be back next week, he may have a better idea on why this is happening.
Sascha, Can you trace DSF-GDB? you could start from the following method and see the area where this MI command is being created/triggered. CommandFactory.java#createMIBreakDelete Doing Ctrl+Alt+H while selecting this method in the Editor gives you the callers, and from there it expands pretty quickly, but using break points while on your target could provide some good hints on the function area related to this behavior.
Alvaro, I will try. However, how do I trace Eclipse, are there some instructions on how to do that? Sorry, never done that before.
Sascha, here are some instructions on how to set up Eclipse and get started with the CDT code: http://wiki.eclipse.org/Getting_started_with_CDT_development
Marc-Andre, thanks for the instructions. However, I am running Eclipse on a Linux server without Internet access. But it looks like the instructions require access to the Internet to download code from git and also install the required dependencies.
Ok, so I am not sure I will be able to reproduce the issue on my Linux machine. But I am willing to try out suggestions. Can anybody suggest some features to try enable/disable to see if that makes a difference? Under what conditions break-delete is called when brining up the debug session?
FYI, I was able to remotely debug the Eclipse instance with the breakpoint issue, but for some reason I was not able to set successfully set a breakpoint, so still don't know where break-delete gets called from.
By not successfully being able to set a breakpoint, I am talking about remote breakpoints for the Eclipse CDT instance being debugger. For some reason, I couldn't get Eclipse CDT to stop at any remote breakpoint.
My guess is that this is related to the change of handling in the thread filtering that I did in Bug 431986. For example for GDB 7.0 and 7.1, breakpoints get deleted right away (Bug 435144). In the eclipse gdb console, can you type show version and provide us with the output? The 'gdb console' can be made easily visible if you make the console view visible and then select the 'gdb' element in the debug view. Also, can you attach the entire 'gdb traces' output to this bug? You can save it to a file using the 'save' button from the 'gdb traces' console. The snippets you provided don't have all the commands I was expecting to see. Thanks
This is the output from 'show version': show version GNU gdb (GDB) 7.6 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-unknown-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Unfortunately, can't attach the entire output, since they may be some proprietary stuff in there. Just let me know what you are looking for and I will see.
(In reply to Sascha Konrad from comment #14) > Unfortunately, can't attach the entire output, since they may be some > proprietary stuff in there. Just let me know what you are looking for and I > will see. Same as what you have in comment #1 except: 1- if you removed any lines, please indicate so and put the corresponding timestamp and MI command itself as well as it's answer (without the details is ok) so we can know the command was sent, and its corresponding answer was received. Similarly for MI events. 2- Can you add a couple of lines of output after the last -break-delete just so we can see that nothing related is happening. Thanks
(In reply to Sascha Konrad from comment #12) > By not successfully being able to set a breakpoint, I am talking about > remote breakpoints for the Eclipse CDT instance being debugger. For some > reason, I couldn't get Eclipse CDT to stop at any remote breakpoint. Do you have the same version of the code in the debugging machine as well as in the target machine ?
(In reply to Alvaro Sanchez-Leon from comment #16) > (In reply to Sascha Konrad from comment #12) > > By not successfully being able to set a breakpoint, I am talking about > > remote breakpoints for the Eclipse CDT instance being debugger. For some > > reason, I couldn't get Eclipse CDT to stop at any remote breakpoint. > > Do you have the same version of the code in the debugging machine as well as > in the target machine ? I think so. What I did is to copy the source and binary jars from the CDT 8.4.0 zip into my Eclipse CDT 4.4.1 plugins directory. Then I brought up Eclipse CDT, and remotely connected with Eclipse Java 4.4.1 to the process. I imported the Eclipse CDT jars Eclipse Java projects first using Import -> Plugins/Fragments. So it should see the same version of the CDT code that is currently running in Eclipse CDT. But for some reason, it doesn't stop at the breakpoints I set. I can stop the remote process manually though.
Created attachment 246748 [details] Gdb trace output
(In reply to Marc Khouzam from comment #15) > (In reply to Sascha Konrad from comment #14) > > > Unfortunately, can't attach the entire output, since they may be some > > proprietary stuff in there. Just let me know what you are looking for and I > > will see. > > Same as what you have in comment #1 except: > > 1- if you removed any lines, please indicate so and put the corresponding > timestamp and MI command itself as well as it's answer (without the details > is ok) so we can know the command was sent, and its corresponding answer was > received. Similarly for MI events. > > 2- Can you add a couple of lines of output after the last -break-delete just > so we can see that nothing related is happening. > > Thanks Marc, attached another sanitized trace to this bug that shows the situation where a few breakpoints get deleted again even though they were inserted successfully in the first place. This is pretty much complete trace from when the session is started until it stops in main.
Ok, I started over from a clean slate, and this time remote debugging worked. I was finally able to catch the erroneous delete call in the act, here is the stack trace: CommandFactory_6_8(CommandFactory).createMIBreakDelete(IBreakpointsTargetDMContext, int[]) line: 374 MIBreakpoints$12.execute(RequestMonitor) line: 1313 MIRunControl$8(Sequence).executeStep(int) line: 459 MIRunControl$8(Sequence).run() line: 363 Executors$RunnableAdapter<T>.call() line: 511 ScheduledThreadPoolExecutor$ScheduledFutureTask<V>(FutureTask<V>).run() line: 266 ScheduledThreadPoolExecutor$ScheduledFutureTask<V>.access$201(ScheduledThreadPoolExecutor$ScheduledFutureTask) line: 180 ScheduledThreadPoolExecutor$ScheduledFutureTask<V>.run() line: 293 DefaultDsfExecutor(ThreadPoolExecutor).runWorker(ThreadPoolExecutor$Worker) line: 1142 ThreadPoolExecutor$Worker.run() line: 617 Thread.run() line: 745
Btw, is there a temporary workaround anybody can suggest to prevent this from happening? Do we have a clue now what the issue is or do we need some more information?
Bump! Anybody? Was hoping we could still get a fix into 4.4.1. If there is some information required here, please let me know.
Correlating the stack trace and the gdb traces, it looks like MIBreakpointsManager#uninstallBreakpoint() is being called. I cannot be sure from the info I have, but if I am to guess the path leading to that call, I'd go for MIBreakpointsManager#modifyBreakpoint() which calls uninstallBreakpoint() if we find that the breakpoint is (incorrectly?) filtered. modifyBreakpoint() makes sense as I've seen it being called when the install count is incremented, which does happen here. I'm going to have to dig deeper, looking for some scenario that could lead to that. One suspicious thing I see is that there is a 4 seconds pause in the gdb traces, before the breakpoints get deleted. In fact what happens in the traces is: 1- break-insert for 6 breakpoints 2- 3 of them get created right away 3- 4 seconds of nothing happening 4- only the first three created breakpoints get deleted interleaved with the answer to the creation of the other three bp Why are only the bp created before the pause deleted? Is there some timeout or something? What is that pause, which seems to be caused on the gdb side?
Thanks Marc. To me it looks like another thread kicking in, possibly a timeout, that then ends up deleting the breakpoint. And yes, when bringing up the task for debugging there is a couple of seconds of delay, not sure where it's coming from. Sometimes, some of the checkmarks next to the breakpoints are missing initially, but after a few seconds they show up. So sometimes a breakpoint gets deleted, but a couple of seconds later it gets set again. Also, the less idle the machine has and the longer it takes for the debug session to come up, the more aggravated the problem seems to be. (In reply to Marc Khouzam from comment #23) > Correlating the stack trace and the gdb traces, it looks like > MIBreakpointsManager#uninstallBreakpoint() is being called. > > I cannot be sure from the info I have, but if I am to guess the path leading > to that call, I'd go for MIBreakpointsManager#modifyBreakpoint() which calls > uninstallBreakpoint() if we find that the breakpoint is (incorrectly?) > filtered. modifyBreakpoint() makes sense as I've seen it being called when > the install count is incremented, which does happen here. > > I'm going to have to dig deeper, looking for some scenario that could lead > to that. > > One suspicious thing I see is that there is a 4 seconds pause in the gdb > traces, before the breakpoints get deleted. In fact what happens in the > traces is: > > 1- break-insert for 6 breakpoints > 2- 3 of them get created right away > 3- 4 seconds of nothing happening > 4- only the first three created breakpoints get deleted interleaved with the > answer to the creation of the other three bp > > Why are only the bp created before the pause deleted? Is there some timeout > or something? What is that pause, which seems to be caused on the gdb side?
(In reply to Sascha Konrad from comment #20) > Ok, I started over from a clean slate, and this time remote debugging > worked. I was finally able to catch the erroneous delete call in the act, > here is the stack trace: > It's pretty good that you can remote debug, the bad thing is that it make take several iterations to back track a meaningful area. The stack trace you got is too deep and has gone through a DSF executor which prevents us from seeing the actual caller. You could start adding breakpoints to e.g. CommandFactory.java#createMIBreakDelete or MIBreakpoints#removeBreakpoint or as Marc suggested: MIBreakpointsManager#modifyBreakpoint Once you find the immediate caller you will need to repeat the process from the top line of the caller to find the next one, this is necessary due to the async calls returning to the handling request monitor via an executor. The observations from Marc are pretty relevant it would be great to find some theories to explore around those questions.
I just saw something similar in my own environment. I created a breakpoint after the debug session was launched, and it got deleted right away! But I can't get it to happen again. I'll try to use the JUnit tests to trigger it again.
(In reply to Marc Khouzam from comment #26) > I just saw something similar in my own environment. I created a breakpoint > after the debug session was launched, and it got deleted right away! But I > can't get it to happen again. I'll try to use the JUnit tests to trigger it > again. Thanks, I will see if I can create some more debug traces too. In order to reproduce the problem Marc, try to make bringing up the debug session slow by running some background tasks that utilize all your cores and make the machine busy. Some IO heavy tasks may help too. Also, create a large number of breakpoints, the more breakpoints the more likely also the occurrence of the issue.
Sascha, did you ever check your .log file to see if there are any exceptions or something? It is in <yourWorkspace>/.metadata/.log
(In reply to Marc Khouzam from comment #28) > Sascha, did you ever check your .log file to see if there are any exceptions > or something? It is in > > <yourWorkspace>/.metadata/.log Just happened again and I checked the log. No exception or anything else in there.
Ok, I set the following breakpoints: BreakpointManager [line: 604] - deleteMIBreakpoints(MISession, int[]) CommandFactory [line: 56] - createMIBreakDelete(int[]) CommandFactory [line: 374] - createMIBreakDelete(IBreakpointsTargetDMContext, int[]) MIBreakpoints [line: 905] - removeBreakpoint(IBreakpointDMContext, RequestMonitor) MIBreakpoints [line: 1026] - modifyBreakpoint(IBreakpointDMContext, Map<String, Object>, RequestMonitor, boolean) Here are three different stack traces I saw: 1. GDBBreakpoints_7_6(MIBreakpoints).removeBreakpoint(IBreakpoints$IBreakpointDMContext, RequestMonitor) line: 905 MIBreakpointsManager$12.handleCompleted() line: 810 RequestMonitor$2.run() line: 303 Executors$RunnableAdapter<T>.call() line: 511 ScheduledThreadPoolExecutor$ScheduledFutureTask<V>(FutureTask<V>).run() line: 266 ScheduledThreadPoolExecutor$ScheduledFutureTask<V>.access$201(ScheduledThreadPoolExecutor$ScheduledFutureTask) line: 180 ScheduledThreadPoolExecutor$ScheduledFutureTask<V>.run() line: 293 DefaultDsfExecutor(ThreadPoolExecutor).runWorker(ThreadPoolExecutor$Worker) line: 1142 ThreadPoolExecutor$Worker.run() line: 617 Thread.run() line: 745 2. CommandFactory_6_8(CommandFactory).createMIBreakDelete(IBreakpointsTargetDMContext, int[]) line: 374 MIBreakpoints$12.execute(RequestMonitor) line: 1313 MIRunControl$8(Sequence).executeStep(int) line: 459 MIRunControl$8(Sequence).run() line: 363 Executors$RunnableAdapter<T>.call() line: 511 ScheduledThreadPoolExecutor$ScheduledFutureTask<V>(FutureTask<V>).run() line: 266 ScheduledThreadPoolExecutor$ScheduledFutureTask<V>.access$201(ScheduledThreadPoolExecutor$ScheduledFutureTask) line: 180 ScheduledThreadPoolExecutor$ScheduledFutureTask<V>.run() line: 293 DefaultDsfExecutor(ThreadPoolExecutor).runWorker(ThreadPoolExecutor$Worker) line: 1142 ThreadPoolExecutor$Worker.run() line: 617 Thread.run() line: 745 3. GDBBreakpoints_7_6(MIBreakpoints).modifyBreakpoint(IBreakpointDMContext, Map<String,Object>, RequestMonitor, boolean) line: 1026 GDBBreakpoints_7_0.access$3(GDBBreakpoints_7_0, IBreakpoints$IBreakpointDMContext, Map, RequestMonitor, boolean) line: 1 GDBBreakpoints_7_0$4.handleCompleted() line: 289 RequestMonitor$2.run() line: 303 Executors$RunnableAdapter<T>.call() line: 511 ScheduledThreadPoolExecutor$ScheduledFutureTask<V>(FutureTask<V>).run() line: 266 ScheduledThreadPoolExecutor$ScheduledFutureTask<V>.access$201(ScheduledThreadPoolExecutor$ScheduledFutureTask) line: 180 ScheduledThreadPoolExecutor$ScheduledFutureTask<V>.run() line: 293 DefaultDsfExecutor(ThreadPoolExecutor).runWorker(ThreadPoolExecutor$Worker) line: 1142 ThreadPoolExecutor$Worker.run() line: 617 Thread.run() line: 745
(In reply to Sascha Konrad from comment #30) > Ok, I set the following breakpoints: Thanks for doing this. > BreakpointManager [line: 604] - deleteMIBreakpoints(MISession, int[]) > CommandFactory [line: 56] - createMIBreakDelete(int[]) > CommandFactory [line: 374] - > createMIBreakDelete(IBreakpointsTargetDMContext, int[]) > MIBreakpoints [line: 905] - removeBreakpoint(IBreakpointDMContext, > RequestMonitor) > MIBreakpoints [line: 1026] - modifyBreakpoint(IBreakpointDMContext, > Map<String, Object>, RequestMonitor, boolean) > > Here are three different stack traces I saw: > 2. > CommandFactory_6_8(CommandFactory). > createMIBreakDelete(IBreakpointsTargetDMContext, int[]) line: 374 > MIBreakpoints$12.execute(RequestMonitor) line: 1313 Sadly this does not give us any info. > 3. > GDBBreakpoints_7_6(MIBreakpoints).modifyBreakpoint(IBreakpointDMContext, > Map<String,Object>, RequestMonitor, boolean) line: 1026 > GDBBreakpoints_7_0.access$3(GDBBreakpoints_7_0, > IBreakpoints$IBreakpointDMContext, Map, RequestMonitor, boolean) line: 1 > GDBBreakpoints_7_0$4.handleCompleted() line: 289 This does not seem to lead to a delete, so it looks like a valid call. > 1. > GDBBreakpoints_7_6(MIBreakpoints). > removeBreakpoint(IBreakpoints$IBreakpointDMContext, RequestMonitor) line: > 905 > MIBreakpointsManager$12.handleCompleted() line: 810 This is the call that is causing problems. Looking at what can call this, the most likely is MIBreakpointsManager#breakpointChanged() calling MIBreakpointsManager#modifyBreakpoint() which is expected, but then it calls MIBreakpointsManager#uninstallBreakpoint() which would be a mistake. To confirm this, you could set a breakpoint a line 904 of MIBreakpointsManager. It is the call to MIBreakpointsManager#uninstallBreakpoint()that looks like this: if (filtered) { uninstallBreakpoint(dmc, breakpoint, rm ); return; } I'm going to look into the code to see what could make us think the bp should be filtered. Another thing to look at is the breakpoint properties for a breakpoint that was mistakenly removed (right-click on that bp in the bp view). On the left bar you should see 'Actions', "Common", and "Filter", choose the 'filter' page and see what is shown.
If I'm right about the bad code path, the method of interest will be MIBreakpointsManager#isBreakpointEntirelyFiltered() The question will be what is the value of 'bpTargetDmc' and the different values of 'procDmc' do we can see why they are never equal and that method returns true. Sascha what you could do is modify MIBreakpointsManager#isBreakpointEntirelyFiltered() to add the following printout: System.out.println("Comparing procDmc: " + procDmc.toString() + " with bpTargetDmc: " + bpTargetDmc.toString()); please add that printout as the first line of the for loop in the method. Then, set the breakpoint at MIBreakpointsManager#uninstallBreakpoint()that looks like this: if (filtered) { uninstallBreakpoint(dmc, breakpoint, rm ); return; } and if it hits, collect the printouts saying "Comparing..." that were printed right before the bp hit.
Ok, here some more stack traces: MIBreakpointsManager.isBreakpointEntirelyFiltered(IBreakpoints$IBreakpointsTargetDMContext, ICBreakpoint) line: 2079 MIBreakpointsManager.modifyBreakpoint(IBreakpointsTargetDMContext, ICBreakpoint, Map<String,Object>, IMarkerDelta, RequestMonitor) line: 875 MIBreakpointsManager.access$17(MIBreakpointsManager, IBreakpoints$IBreakpointsTargetDMContext, ICBreakpoint, Map, IMarkerDelta, RequestMonitor) line: 859 MIBreakpointsManager$21$2.handleSuccess() line: 1306 MIBreakpointsManager$21$2(RequestMonitor).handleCompleted() line: 376 RequestMonitor$2.run() line: 303 Executors$RunnableAdapter<T>.call() line: 511 ScheduledThreadPoolExecutor$ScheduledFutureTask<V>(FutureTask<V>).run() line: 266 ScheduledThreadPoolExecutor$ScheduledFutureTask<V>.access$201(ScheduledThreadPoolExecutor$ScheduledFutureTask) line: 180 ScheduledThreadPoolExecutor$ScheduledFutureTask<V>.run() line: 293 DefaultDsfExecutor(ThreadPoolExecutor).runWorker(ThreadPoolExecutor$Worker) line: 1142 ThreadPoolExecutor$Worker.run() line: 617 Thread.run() line: 745 I do indedd see MIBreakpointsManager#uninstallBreakpoint() being called from line 904: MIBreakpointsManager.modifyBreakpoint(IBreakpointsTargetDMContext, ICBreakpoint, Map<String,Object>, IMarkerDelta, RequestMonitor) line: 904 MIBreakpointsManager.access$17(MIBreakpointsManager, IBreakpoints$IBreakpointsTargetDMContext, ICBreakpoint, Map, IMarkerDelta, RequestMonitor) line: 859 MIBreakpointsManager$21$2.handleSuccess() line: 1306 MIBreakpointsManager$21$2(RequestMonitor).handleCompleted() line: 376 RequestMonitor$2.run() line: 303 Executors$RunnableAdapter<T>.call() line: 511 ScheduledThreadPoolExecutor$ScheduledFutureTask<V>(FutureTask<V>).run() line: 266 ScheduledThreadPoolExecutor$ScheduledFutureTask<V>.access$201(ScheduledThreadPoolExecutor$ScheduledFutureTask) line: 180 ScheduledThreadPoolExecutor$ScheduledFutureTask<V>.run() line: 293 DefaultDsfExecutor(ThreadPoolExecutor).runWorker(ThreadPoolExecutor$Worker) line: 1142 ThreadPoolExecutor$Worker.run() line: 617 Thread.run() line: 745 Regarding the filtering, all breakpoints (removed or not) seem to have the same filter, restricted to a selected process/thread with the only thread in my single threaded application being checked. However, I noticed something interesting. The breakpoints that were removed again, if I click on "Show Breakpoints Supported by Selected Target" button, they are actually hidden and displayed again. So it looks like breakpoints are being removed again as it thinks it's not supported by target.
Ok, I was not able to put the print statement in the isBreakpointEntirelyFiltered function, but I was able to step through it in the debugger. What seems to happen when it returns true is that getFilterExtension has returned an empty array. When getFilterExtension returns an array with values, it always seems to find a matching value in procTargets and then returns false. Here the code I am talking about: private boolean isBreakpointEntirelyFiltered(IBreakpointsTargetDMContext bpTargetDmc, ICBreakpoint breakpoint) { try { IContainerDMContext[] procTargets = getFilterExtension(breakpoint).getTargetFilters(); for (IContainerDMContext procDmc : procTargets) { if (procDmc.equals(bpTargetDmc) || DMContexts.isAncestorOf(procDmc, bpTargetDmc)) { return false; } } } catch (CoreException e) { } return true; }
(In reply to Sascha Konrad from comment #34) > Ok, I was not able to put the print statement in the > isBreakpointEntirelyFiltered function, but I was able to step through it in > the debugger. What seems to happen when it returns true is that > getFilterExtension has returned an empty array. When getFilterExtension > returns an array with values, it always seems to find a matching value in > procTargets and then returns false. Great work, this is new info that points to something I haven't encountered before. I'll look into it and see if I can figure out what is happening.
(In reply to Sascha Konrad from comment #33) > Regarding the filtering, all breakpoints (removed or not) seem to have the > same filter, restricted to a selected process/thread with the only thread in > my single threaded application being checked. Are you sure that the removed bp have the process selected in the filter window? That is unexpected to me. I would expect nothing to be selected for those bps. > However, I noticed something interesting. The breakpoints that were removed > again, if I click on "Show Breakpoints Supported by Selected Target" button, > they are actually hidden and displayed again. So it looks like breakpoints > are being removed again as it thinks it's not supported by target. This would make sense if the filter window does not show the process or thread as selected, as it thinks the bp is filtered from that process. But your sentence above does not seem to agree...
(In reply to Marc Khouzam from comment #36) > (In reply to Sascha Konrad from comment #33) > > > Regarding the filtering, all breakpoints (removed or not) seem to have the > > same filter, restricted to a selected process/thread with the only thread in > > my single threaded application being checked. > > Are you sure that the removed bp have the process selected in the filter > window? That is unexpected to me. I would expect nothing to be selected > for those bps. > > > However, I noticed something interesting. The breakpoints that were removed > > again, if I click on "Show Breakpoints Supported by Selected Target" button, > > they are actually hidden and displayed again. So it looks like breakpoints > > are being removed again as it thinks it's not supported by target. > > This would make sense if the filter window does not show the process or > thread as selected, as it thinks the bp is filtered from that process. But > your sentence above does not seem to agree... Well, these two things may have been observed during different runs, where one of them may not have had any breakpoints removed. Will go back and check if both of these are true on the same run.
(In reply to Marc Khouzam from comment #36) > (In reply to Sascha Konrad from comment #33) > > > Regarding the filtering, all breakpoints (removed or not) seem to have the > > same filter, restricted to a selected process/thread with the only thread in > > my single threaded application being checked. > > Are you sure that the removed bp have the process selected in the filter > window? That is unexpected to me. I would expect nothing to be selected > for those bps. > > > However, I noticed something interesting. The breakpoints that were removed > > again, if I click on "Show Breakpoints Supported by Selected Target" button, > > they are actually hidden and displayed again. So it looks like breakpoints > > are being removed again as it thinks it's not supported by target. > > This would make sense if the filter window does not show the process or > thread as selected, as it thinks the bp is filtered from that process. But > your sentence above does not seem to agree... Hi Marc, Just checked again: Both deleted as well as active breakpoints both have the same process selected in the Filter properties. I don't see a difference between these two regarding the filter. However, clicking on "Show Breakpoints Supported..." button hides only the ones that were deleted, so the filter has no impact here.
(In reply to Sascha Konrad from comment #38) > Just checked again: Both deleted as well as active breakpoints both have the > same process selected in the Filter properties. I don't see a difference > between these two regarding the filter. This is strange. Maybe the key to the problem... I have more thinking to do. > However, clicking on "Show Breakpoints Supported..." button hides only the > ones that were deleted, so the filter has no impact here.
(In reply to Marc Khouzam from comment #39) > (In reply to Sascha Konrad from comment #38) > > > Just checked again: Both deleted as well as active breakpoints both have the > > same process selected in the Filter properties. I don't see a difference > > between these two regarding the filter. > > This is strange. > Maybe the key to the problem... > I have more thinking to do. > > > However, clicking on "Show Breakpoints Supported..." button hides only the > > ones that were deleted, so the filter has no impact here. Ok, maybe to help you reproduce it, this is definitely some timing issue. When I bring up the debug sessions with my remote breakpoints set and step through them, I end up with almost all of the CDT breakpoints being deleted again. However, if I run the debug session with "Skip All Breakpoints" turned on in the Java debug instance, then I end up with only a few or none of the CDT breakpoints being deleted. This also coincides with my observation that the problem is exacerbated when the machine is busy. Here my breakpoints, see if you can set them and stop in them, and thereby reproduce the issue: BreakpointManager [line: 604] - deleteMIBreakpoints(MISession, int[]) CommandFactory [line: 56] - createMIBreakDelete(int[]) CommandFactory [line: 374] - createMIBreakDelete(IBreakpointsTargetDMContext, int[]) MIBreakpoints [line: 905] - removeBreakpoint(IBreakpointDMContext, RequestMonitor) MIBreakpoints [line: 1026] - modifyBreakpoint(IBreakpointDMContext, Map<String, Object>, RequestMonitor, boolean) MIBreakpointsManager [line: 904] - modifyBreakpoint(IBreakpointsTargetDMContext, ICBreakpoint, Map<String, Object>, IMarkerDelta, RequestMonitor) MIBreakpointsManager [line: 2079] - isBreakpointEntirelyFiltered(IBreakpointsTargetDMContext, ICBreakpoint) MIBreakpointsManager [line: 2081] - isBreakpointEntirelyFiltered(IBreakpointsTargetDMContext, ICBreakpoint) MIBreakpointsManager [line: 2087] - isBreakpointEntirelyFiltered(IBreakpointsTargetDMContext, ICBreakpoint)
(In reply to Sascha Konrad from comment #40) > Here my breakpoints, see if you can set them and stop in them, and thereby > reproduce the issue: Thanks, I will try that. Another question. After a breakpoint gets deleted, can you get it to be installed again by playing with the filter? Disable the filter, close the dialog, then re-open the dialog and re-enable the filter.
If you restart eclipse, after that, does the problem happen the very first time you launch? Or does it only happen the second time and the times after that?
Ok, I finally was able to reproduce it when putting load on the machine. In my case some bp get deleted, but then they get installed again. But still, I do see the empty array returned by getTargetFilters().
It seems that we have a race condition between the processing of ^done when a bp is created, and the =thread-group-started event indicating the process is started. When we get ^done, we increment the install count of the bp. This causes the platform to call MIBreakpointsManager#breakpointChanged() which calls modifyBreakpoint(). modifyBreakpoint() checks if the bp is filtered or not. When the machine is slow, this happens *before* we get the IStartedDMEvent that indicates the process is running, which we use to properly initialize the filtering. So, it looks like the bp is filetered and it gets deleted. I'm thinking of a fix for this. Thanks Sascha for your help and patience!
(In reply to Marc Khouzam from comment #44) > It seems that we have a race condition between the processing of ^done when > a bp is created, and the =thread-group-started event indicating the process > is started. > > When we get ^done, we increment the install count of the bp. This causes > the platform to call MIBreakpointsManager#breakpointChanged() which calls > modifyBreakpoint(). modifyBreakpoint() checks if the bp is filtered or not. > When the machine is slow, this happens *before* we get the IStartedDMEvent > that indicates the process is running, which we use to properly initialize > the filtering. So, it looks like the bp is filetered and it gets deleted. Marc, shouldn't the initial breakpoints be installed when IStartedDMEvent is received? How it would work with multiple processes?
(In reply to Mikhail Khodjaiants from comment #45) > Marc, shouldn't the initial breakpoints be installed when IStartedDMEvent is > received? How it would work with multiple processes? IStartedDMEvent indicates that the process has started to *run*, while we set breakpoints when the binary is loaded. So, in DebugNewProcessSequence, we load the binary, then set the breakpoints, and then start the program. We couldn't run the program first as it may miss the bp that are not set yet. Same logic works for multi-process. This shows a bit the issue we are facing with the filtering, where we wait for the process to run to set the filter, when all we need is to know that we are about to debug that process; it does not actually have to be running.
(In reply to Marc Khouzam from comment #44) > Thanks Sascha for your help and patience! Great, no problem! :-) Glad we finally found the root cause of this annoying issue that is plaguing me. Hope we can get a fix in for SR1. Fingers crossed!
(In reply to Marc Khouzam from comment #46) > (In reply to Mikhail Khodjaiants from comment #45) > > > Marc, shouldn't the initial breakpoints be installed when IStartedDMEvent is > > received? How it would work with multiple processes? > > IStartedDMEvent indicates that the process has started to *run*, while we > set breakpoints when the binary is loaded. So, in DebugNewProcessSequence, > we load the binary, then set the breakpoints, and then start the program. > We couldn't run the program first as it may miss the bp that are not set > yet. Same logic works for multi-process. > > This shows a bit the issue we are facing with the filtering, where we wait > for the process to run to set the filter, when all we need is to know that > we are about to debug that process; it does not actually have to be running. It is actually even more complicated with GDB >= 7.4. For these newer GDBs we only set the breakpoints once even for multi-process, but we still need to add each new process created to the filter list. We were using the IStartedDMEvent to know a new process was created, but it looks like we will need a more direct call to deal with the filters.
I pushed a fix to Gerrit: https://git.eclipse.org/r/33393 The idea is that as soon as we start tracking the breakpoints, we initialize the filters. This is how we used to do things before bug 431986, in fact, we used to initialize the filter directly in installInitialBreakpoints() which is called by startTrackingBreakpoints(). The reason this changed is that the context passed to startTrackingBreakpoints() is an IBreakpointsTargetDMContext which no longer holds the containerDmc; instead, I used the IStartedDMEvent. The new solution creates a new API MIBreakpointsManager#initTargetFilterForBreakpoints() which takes an IContainerDMC as a parameter. This method is now called right after startTrackingBreakpoints(), and initializes the filter immediately using the container. I'm hesitant to put this in 8.5 though. We have no soak time and if I missed something important, it touches the essential feature of breakpoints. Marc-Andre pointed out that this bug has been in the code base since March and was only noticed now, so it may not be as urgent is it seems. Sascha, could you install a CDT from a nightly build update site to get the fix so you are not blocked? I can provide one for you.
The more I think about it the more I realize I can't put this in 8_5. I've only tried the fix in my limited environment with a specific bug in mind. This fix affects setting breakpoints at launch time, we just can't risk breaking that. I'll cherry pick the fix for master instead. Sascha, you worked hard to get this fixed so I will do my best to provide you with an easy way to get the fix directly.
(In reply to Marc Khouzam from comment #50) > Sascha, you worked hard to get this fixed so I will do my best to provide > you with an easy way to get the fix directly. Thanks! I agree, let's not rush it out and break something else. Will test it out in my environment once it's available.
That actually raises another question: Why does it take so long on my machine to launch GDB? Looking at the progress view, the step "Configuring GDB" takes about 10 seconds to complete, even when machine is 90% idle. Any suggestion how to debug this?
(In reply to Sascha Konrad from comment #52) > That actually raises another question: Why does it take so long on my > machine to launch GDB? Looking at the progress view, the step "Configuring > GDB" takes about 10 seconds to complete, even when machine is 90% idle. > > Any suggestion how to debug this? Actually, I think it's this command here that takes a long time: 946,323 15-list-thread-groups 951,111 14^done Any clue what it's doing and any way to speed it up maybe?
(In reply to Sascha Konrad from comment #53) > (In reply to Sascha Konrad from comment #52) > > That actually raises another question: Why does it take so long on my > > machine to launch GDB? Looking at the progress view, the step "Configuring > > GDB" takes about 10 seconds to complete, even when machine is 90% idle. > > > > Any suggestion how to debug this? > > Actually, I think it's this command here that takes a long time: > > 946,323 15-list-thread-groups > 951,111 14^done > > Any clue what it's doing and any way to speed it up maybe? Never mind, I think it's the previous step that takes that long: file-exec-and-symbols --thread-group i1 xxx.linux_64.tsk When I see the slowdown, need to check if I see the same slow gdb startup outside of Eclipse as well.
(In reply to Sascha Konrad from comment #53) > 946,323 15-list-thread-groups > 951,111 14^done > Never mind, I think it's the previous step that takes that long: > > file-exec-and-symbols --thread-group i1 xxx.linux_64.tsk Right. Notice the number prefixing the commands. It indicates for which command the answer is for. So, above, the ^done is for command #14. > When I see the slowdown, need to check if I see the same slow gdb startup > outside of Eclipse as well. That is the best way to isolate problems like that. And it makes sense that it takes time to load the binary if your system is slow and the binary is big. I'm not sure 5 seconds isn't too slow, but I'm pretty sure this will be gdb's delay.
Marc, I added a comment to https://git.eclipse.org/r/33393 without noticing that it is abandoned. As far as I understand you are planning to submit the same patch for master. If that's correct then consider the patch as reviewed :)
(In reply to Mikhail Khodjaiants from comment #56) > Marc, I added a comment to https://git.eclipse.org/r/33393 without noticing > that it is abandoned. As far as I understand you are planning to submit the > same patch for master. If that's correct then consider the patch as reviewed > :) That is right. Thank you!
I've implemented Mikhail's suggestion for master and pushed to Gerrit: https://git.eclipse.org/r/33458 The new code is simpler than my original proposal as it avoids code duplication. I tested with multi-process, with attach, and with GDB 7.0, 7.3 and 7.8. I'll wait for the HIPP build to finish and then commit to master. Once it is on master, Sascha will be able to easily install it from the HIPP update site and let us know if the problem is fixed. I'll post the p2 repo address once ready.
(In reply to Marc Khouzam from comment #58) > I've implemented Mikhail's suggestion for master and pushed to Gerrit: > https://git.eclipse.org/r/33458 > > The new code is simpler than my original proposal as it avoids code > duplication. > > I tested with multi-process, with attach, and with GDB 7.0, 7.3 and 7.8. > > I'll wait for the HIPP build to finish and then commit to master. > Once it is on master, Sascha will be able to easily install it from the HIPP > update site and let us know if the problem is fixed. I'll post the p2 repo > address once ready. HI Marc, Thanks again for the quick fix. Is the fix available yet somewhere for download so I can test it out? Thanks
(In reply to Sascha Konrad from comment #59) > Is the fix available yet somewhere for download so I can test it out? Thanks Woops, sorry, I forgot to post the link. You can install or upgrade from: https://hudson.eclipse.org/cdt/job/cdt-master/lastSuccessfulBuild/artifact/releng/org.eclipse.cdt.repo/target/repository which is the build of the latest master branch, which contains the fix in question. Please let us know if it fixes your problem. If not, we'll re-open the bug. Thanks!
(In reply to Marc Khouzam from comment #60) > (In reply to Sascha Konrad from comment #59) > > > Is the fix available yet somewhere for download so I can test it out? Thanks > > Woops, sorry, I forgot to post the link. > You can install or upgrade from: > https://hudson.eclipse.org/cdt/job/cdt-master/lastSuccessfulBuild/artifact/ > releng/org.eclipse.cdt.repo/target/repository > > which is the build of the latest master branch, which contains the fix in > question. > > Please let us know if it fixes your problem. If not, we'll re-open the bug. > > Thanks! Looks like my current Eclipse 4.4.1 comes with CDT 8.4.0. Can I just install CDT 8.5.0 instead?
(In reply to Sascha Konrad from comment #61) > Looks like my current Eclipse 4.4.1 comes with CDT 8.4.0. Can I just install > CDT 8.5.0 instead? CDT 8.5.0 won't have the fix, that is why you need to use this non-released build: https://hudson.eclipse.org/cdt/job/cdt-master/lastSuccessfulBuild/artifact/releng/org.eclipse.cdt.repo/target/repository You can install that CDT on Eclipse 4.4 or 4.4.1
(In reply to Marc Khouzam from comment #62) > (In reply to Sascha Konrad from comment #61) > > > Looks like my current Eclipse 4.4.1 comes with CDT 8.4.0. Can I just install > > CDT 8.5.0 instead? > > CDT 8.5.0 won't have the fix, that is why you need to use this non-released > build: > > https://hudson.eclipse.org/cdt/job/cdt-master/lastSuccessfulBuild/artifact/ > releng/org.eclipse.cdt.repo/target/repository > > You can install that CDT on Eclipse 4.4 or 4.4.1 Hi Marc, I just installed the update from the repository above into a clean install of Eclipse 4.4.1 (the one that just got released). I also installed all the optional features that are there, at least that ones that didn't have missing dependencies. But now I can't set a breakpoint anymore, it just tells me "The chosen operation is not enabled." Any advice how to fix that? Thanks.
(In reply to Sascha Konrad from comment #63) > (In reply to Marc Khouzam from comment #62) > > (In reply to Sascha Konrad from comment #61) > > > > > Looks like my current Eclipse 4.4.1 comes with CDT 8.4.0. Can I just install > > > CDT 8.5.0 instead? > > > > CDT 8.5.0 won't have the fix, that is why you need to use this non-released > > build: > > > > https://hudson.eclipse.org/cdt/job/cdt-master/lastSuccessfulBuild/artifact/ > > releng/org.eclipse.cdt.repo/target/repository > > > > You can install that CDT on Eclipse 4.4 or 4.4.1 > > Hi Marc, > > I just installed the update from the repository above into a clean install > of Eclipse 4.4.1 (the one that just got released). > > I also installed all the optional features that are there, at least that > ones that didn't have missing dependencies. > > But now I can't set a breakpoint anymore, it just tells me "The chosen > operation is not enabled." > > Any advice how to fix that? Thanks. Never mind, looks like this error is caused by PyDev rather than CDT. PyDev not yet compatible with 4.4.1 it seems, after removing PyDev it works fine again.
Did some quick testing today, seems okay. Final question Marc: Will this patch automatically become part of the Eclipse Luna SR2/CDT 8.6.0 release now whenever it gets released? Thanks!
(In reply to Sascha Konrad from comment #65) > Did some quick testing today, seems okay. Great! Thanks. > Final question Marc: Will this patch automatically become part of the > Eclipse Luna SR2/CDT 8.6.0 release now whenever it gets released? Yep.
Sorry, I need to reopen this, this still doesn't appear to be fully fixed. I am running CDT 8.5.0.201409231004 in Eclipse 4.4.1, and when I open a large application for debugging where it takes the gdb several seconds (5-10 secs) to come up, it still keeps happening. :-( 306,722 (gdb) 306,726 23-break-insert -f xxx.cpp:116 306,726 24-break-insert -f yyy.tcc:134 307,229 23^done,bkpt={number="1",type="breakpoint",disp="keep",enabled="y",addr="0x00000000005243ce"\ ,func="xxx_Test::TestBody()",file="xxx.cpp",fullname="xxx.cpp",line="116",thread-groups=["i1"],times="0",original-location="xxx.cpp:116"} 307,229 (gdb) 307,334 25-break-delete 1 <========= 318,814 24^done,bkpt={number="2", ...
Not sure if that is different now, but previously I could disable/re-enable a breakpoint for it to become active (with checkmark). Now it doesn't seem to become active in that case either. I have to completely remove the breakpoint and set it again for it to be active.
Yep, definitely still happening. Marc, do you want me to step through this in the debugger again, or do you have a good idea where the issue is? Another trace, just two breakpoints and the first one gets deleted: ... 994,178 (gdb) 994,183 23-break-insert -f xxx.h:95 994,183 24-break-insert -f yyy.cpp:1021 994,822 23^done,bkpt={number="1",type="breakpoint",disp="keep",enabled="y",... 994,823 (gdb) 994,987 25-break-delete 1 995,419 24^done,bkpt={number="2",type="breakpoint",disp="keep",enabled="y",... 995,419 (gdb) 995,420 25^done 995,420 (gdb) 995,425 26-break-insert -t -f main ...
(In reply to Sascha Konrad from comment #69) > Yep, definitely still happening. > > Marc, do you want me to step through this in the debugger again, or do you > have a good idea where the issue is? Alvaro, do you have time to look at this?
> > Alvaro, do you have time to look at this? I will try to reproduce it first.. Let me know in case you know of any other trigger besides opening a large application (taking 5-10 sec for GDB to come up).
(In reply to Alvaro Sanchez-Leon from comment #71) > > > > Alvaro, do you have time to look at this? > > I will try to reproduce it first.. > > Let me know in case you know of any other trigger besides opening a large > application (taking 5-10 sec for GDB to come up). Try setting the aforementioned breakpoints in CDT in a remote debug session. Stopping at these breakpoints for a couple of seconds, I was able to reproduce it earlier every time.
(In reply to Sascha Konrad from comment #72) > (In reply to Alvaro Sanchez-Leon from comment #71) > > > > > > Alvaro, do you have time to look at this? > > > > I will try to reproduce it first.. > > > > Let me know in case you know of any other trigger besides opening a large > > application (taking 5-10 sec for GDB to come up). > > Try setting the aforementioned breakpoints in CDT in a remote debug session. > Stopping at these breakpoints for a couple of seconds, I was able to > reproduce it earlier every time. Also, set a large number of breakpoints in the C++ application. The more breakpoints you have, the more likely you are to observe the behavior for some of them.
I have spent a good time trying to reproduce this issue without luck ! I have used gdb and gnomecalc as the debug apps, I have setup a VM to establish a remote session that uploads and executes these programs in the server, A good number of breakpoints were setup so they will fill my whole screen and be able to monitor them, and have stopped for few seconds in many of them, to later continue. Could you give me some additional information of your actual launch configuration, any special path mappings, are you using attach-remote or remote launch ?
(In reply to Alvaro Sanchez-Leon from comment #74) > I have spent a good time trying to reproduce this issue without luck ! > > I have used gdb and gnomecalc as the debug apps, > I have setup a VM to establish a remote session that uploads and executes > these programs in the server, > A good number of breakpoints were setup so they will fill my whole screen > and be able to monitor them, and have stopped for few seconds in many of > them, to later continue. > > Could you give me some additional information of your actual launch > configuration, any special path mappings, are you using attach-remote or > remote launch ? Note sure why it happens so frequently in my case. Marc, you were able to reproduce this successfully, can you advise please? I will try to see if I can reproduce this on a clean install and come up with some simple instructions to reproduce.
(In reply to Alvaro Sanchez-Leon from comment #74) > I have spent a good time trying to reproduce this issue without luck ! > > I have used gdb and gnomecalc as the debug apps, > I have setup a VM to establish a remote session that uploads and executes > these programs in the server, > A good number of breakpoints were setup so they will fill my whole screen > and be able to monitor them, and have stopped for few seconds in many of > them, to later continue. > > Could you give me some additional information of your actual launch > configuration, any special path mappings, are you using attach-remote or > remote launch ? Can anyone suggest a nice little open source application that I can use as the test subject? I tried to download and build gnomecalc, but that has some unmet gtk dependencies on my machine. Something non-graphical that I could use reproduce the issue would be nice.
(In reply to Sascha Konrad from comment #76) > Can anyone suggest a nice little open source application that I can use as > the test subject? I tried to download and build gnomecalc, but that has some > unmet gtk dependencies on my machine. Something non-graphical that I could > use reproduce the issue would be nice. Have you tried installing the dependencies for gnomecalc with e.g. > sudo apt-get build-dep gcalctool once you build it, there is also a command line version called gcalccmd that you can try. You could also try gdb itself as a non graphical app.
(In reply to Alvaro Sanchez-Leon from comment #77) > (In reply to Sascha Konrad from comment #76) > > > Can anyone suggest a nice little open source application that I can use as > > the test subject? I tried to download and build gnomecalc, but that has some > > unmet gtk dependencies on my machine. Something non-graphical that I could > > use reproduce the issue would be nice. > > Have you tried installing the dependencies for gnomecalc with e.g. > > sudo apt-get build-dep gcalctool > > once you build it, there is also a command line version called gcalccmd that > you can try. > > You could also try gdb itself as a non graphical app. Unfortunately, this is a Linux server for which I don't have admin rights. I will try to use gdb and see if I get anywhere with it.
Ok, so I was having a hard time initially reproducing this with something else than my test executable. My test executable is about 140 MB (gtest unit test cases on the application code), but the GDB 7.8 executable I built from source is only like 27 MB. So it looks like the size of the executable being debugged definitely is a factor. But I think I finally got somewhere when I involved cpulimit (http://cpulimit.sourceforge.net/). These are the steps to reproduce it with gdb: - Setup gdb source and binary in Eclipse CDT for a debug session (so pretty much use gdb to debug itself). - Create a bunch (like 20) random breakpoints in the gdb source. - Limit CPU of Eclipse process with cpulimit (e.g., cpulimit -p 13856 -l 1) - Launch the gdb debug session. cpulimit should also limit the CPU used by the gdb instance being spawned. - Look at the gdb traces console. When I do that, I see a bunch of deletes happening with cpulimit being active that I don't see without cpulimit: 856,595 52-break-delete 9 856,595 53-break-delete 11 856,595 54-break-delete 21 856,595 55-break-delete 18 856,595 56-break-delete 10 856,595 57-break-delete 13 858,614 50^done 858,614 (gdb) 858,614 51^done 858,614 (gdb) 858,614 58-break-delete 7 858,614 59-break-delete 6 858,614 60-break-delete 12 858,614 61-break-delete 14 Strangely, all the breakpoints seem to get re-inserted again afterwards, but still don't think the deletes should happen in the first place. If you don't see the deletes happen immediately, try stopping and restarting the debug session.
Hi, any luck with reproducing the issue? The use of cpulimit should definitely help. Please note that cpu limit needs to be applied to the PID of the bin/java executable running eclipse to be effective.
Thanks Sasha, It is a big step forward to reproduce the problem, I am a bit short of time at the moment, I think I can try it towards then end of the week. Anybody else in the mean time ?
Thanks for the update. I found somewhat of a workaround in the meantime, where instead of terminating the debug session and re-launching it, I just right-click on it and use Relaunch. Looks like active breakpoints are preserved this way, presumably because gdb never exits. But would still be nice to get this fixed.
Any luck yet? I have a lot of people in my organization asking me about Eclipse, and especially how to use it for debugging. But I am afraid that this bug will turn a lot of them off, and they will discard Eclipse again as something buggy and unreliable. I would like to provide them a nice working installation.
Just to let you know that I have managed to reproduce the problems as you described using cpulimit although I am using a different program. I am using a remote launch so I can apply cpulimit to the application, the gdbserver, the gdbclient and the eclipe runtime, I have started to track the cause of the problem
(In reply to Alvaro Sanchez-Leon from comment #84) > Just to let you know that I have managed to reproduce the problems as you > described using cpulimit although I am using a different program. > > I am using a remote launch so I can apply cpulimit to the application, the > gdbserver, the gdbclient and the eclipe runtime, > > I have started to track the cause of the problem Thanks for the update Alvaro. Glad you can reproduce it now, that's often 90% of solving the problem! :-)
A small update on this issue, Tracing this problem indicates that the issue happens when MIBreakpointsManager receives a call to breakpointChanged from the platform before a call to settingTargetFilter. Setting the target filters comes as a final step in an asynchronous call chain triggered from startTrackingBpForProcess. Discussing this with Marc Khouzam, the first theory is that the asynch calls from the chain mentioned above gets queued in the executor but the executor could also receive the call from breakpointChanged and insert this call before reaching the settingTargetFilter. The theory above sounds pretty good so I experimented with it and made all async calls leading to settingTargetFilter not async by using an ImmediateExcecutor, unfortunately I still see the issue, so the theory above is good but not the final answer. So there is more work to do on this one. I will post more updates as we progress on the issue.
I have submitted a change to Gerrit addressing this issue by setting the target filters before start tracking the break points (suggested by Marc Khouzam) I have tried in my environment and I don't see the issue any more. Sasha could you fetch the following change from Gerrit and give it a try. https://git.eclipse.org/r/#/c/37310/ Cheers
(In reply to Alvaro Sanchez-Leon from comment #87) > I have submitted a change to Gerrit addressing this issue by > setting the target filters before start tracking the break points (suggested > by Marc Khouzam) > > I have tried in my environment and I don't see the issue any more. > > Sasha could you fetch the following change from Gerrit and give it a try. > > https://git.eclipse.org/r/#/c/37310/ > > > Cheers Hi Alvaro, I'd love to try it out. However, is there maybe a zip file I can install to try out? I currently don't have a git repository setup, and I am running Eclipse on a Linux server that doesn't have access to the Internet. If you could provide me a zip file like Marc did in Comment #60, that would be best. Thanks!
> Hi Alvaro, > > I'd love to try it out. However, is there maybe a zip file I can install to > try out? I currently don't have a git repository setup, and I am running > Eclipse on a Linux server that doesn't have access to the Internet. If you > could provide me a zip file like Marc did in Comment #60, that would be > best. Thanks! Since this is not in the master repository yet, there is no nightly build for it, You can build the repository in a local machine with internet access This is straight forward (see [1] below for details) a) clone the repo, e.g. from command line with git://git.eclipse.org/gitroot/cdt/org.eclipse.cdt.git or see 1) on how to do it within Eclipse b) Build CDT and the repo with a single maven command as per [2] Let me know in case you run into problems then I can look for a place to place a generated repo zip file. [1] https://wiki.eclipse.org/Getting_started_with_CDT_development#Building_the_CDT_p2_repository [2] https://wiki.eclipse.org/Getting_started_with_CDT_development#Building_the_CDT_p2_repository
Correction: Link 1) Should be as follows. Thanks Marc-Andre ! > [1] https://wiki.eclipse.org/Getting_started_with_CDT_development
Ok, I was able to build the version from git, apply the patch, transfer the repo to my server, and install it. I was then doing some cursory testing, installing a bunch of breakpoints and running them under the old release as well as the test version from git. With the old release, I was able to reproduce the problem every single time and I saw deletes in the gdb traces. With the new test version, I didn't see the problem anymore, all breakpoints seem to have been set correctly and I didn't see any deletes in the gdb traces. So overall, looks very promising. I am hoping sometime later this week I will be able to put it more through its paces, I will report back then.
(In reply to Sascha Konrad from comment #91) Great to hear !! Looking forward to hear from you, I will request a more detailed review on this correction to make sure we did not miss anything
Hi, just wanted to update that I didn't have all too much time to use it yet, but the few times I did it seemed to work and I didn't notice any issues. I will update here if that changes, but for now it looks like we should proceed with this change, it seem to fix this bug.
(In reply to Alvaro Sanchez-Leon from comment #92) > (In reply to Sascha Konrad from comment #91) > > Great to hear !! > Looking forward to hear from you, > > I will request a more detailed review on this correction to make sure we did > not miss anything Thanks Alvaro!!! Your effort to fix this issue is greatly appreciated.
(In reply to Sascha Konrad from comment #94) > > Thanks Alvaro!!! Your effort to fix this issue is greatly appreciated. The change is now merged into the master branch and ready for CDT 8.6 This has been a nice team effort, Thank you Sascha, your collaboration has made all the difference. Special thanks to Marc Khouzam for his time, support and for proposing this solution.
Hi, Sorry, need to re-open this bug unfortunately! :-( Marc/Alvaro, let me know if you'd rather keep this closed and have a new ticket. So not sure if this is related to our latest changes (I suspect so, but maybe not), didn't have time to test with the old CDT version, but I still see some break-deletes, in a slightly different scenario though. See if you can reproduce it: 1) Set two breakpoints 2) Start the executable to be debugged and stop at a breakpoint 1 765,498 24-break-insert -f xxx.cpp:280 765,498 25-break-insert -f yyy.h:165 3) Now click on breakpoint 2, disabling it (not deleting it) 789,840 36-break-disable 1 4) Select restart from the process's context menu, will stop at breakpoint 1 5) Now try to re-enable the breakpoint 2 that was disabled prior, the little checkmark on the circle disappears and you see this 797,000 48-break-delete 1 797,000 48^done 797,000 (gdb) I think it should be doing a break-enable here, but for some reason it does a break-delete.
(In reply to Sascha Konrad from comment #96) > Hi, Hello Sascha > Sorry, need to re-open this bug unfortunately! :-( Marc/Alvaro, let me know > if you'd rather keep this closed and have a new ticket. It looks like a different problem, it might be related but since the scenario is different a new bug would be best. I tried to reproduce with a simple program from the work space as "C/C++ Application" and the break point delete does not occur. Could you try with a simple Hello program to start and proceed from there? The steps are straight forward so you can hopefully reproduce it with a simple program and simple set up.
(In reply to Alvaro Sanchez-Leon from comment #97) I am now able to see this problem, I have raised the following issue to track this new problem https://bugs.eclipse.org/bugs/show_bug.cgi?id=456959 I will close this bug then ! Thanks Sascha !
Hello the issue is back :(:(:( my trace: 531,912 179-break-insert -f /work/vbe/eclipseLuna64Wrk/9304_vbel_elf2mif/elf2mif.c:372 531,912 179^done,bkpt={number="18",type="breakpoint",disp="keep",enabled="y",addr="0x0000000000400ce\ c",func="main",file="../elf2mif.c",fullname="/work/vbe/eclipseLuna64Wrk/9304_vbel_elf2mif/elf2mif.c"\ ,line="372",times="0",original-location="/work/vbe/eclipseLuna64Wrk/9304_vbel_elf2mif/elf2mif.c:372"\ } 531,912 (gdb) 532,017 180-break-delete 18 532,018 180^done 532,018 (gdb) unfortunately I have 100% repeatability :( I did install latest CDT and still same. My version: Eclipse C/C++ Development Tools Version: 8.6.0.201502131403 I'm using Eclipse Luna I read before it is happening in slow machines. I do have fast and not loaded machine. But I'm connected remotely through NX client - so xfering gui takes some time. Could you please fix the bug?
If someone is breve enough to read down here I have found a workaround by luck that worked for me: instead of: GNU gdb (GDB) CentOS (7.0.1-45.el5.centos) I used: GNU gdb 6.8 Cadence modification version 4.8.3 - 06/2012 And it works now :) Don't know why but works :)
(In reply to vit bernatik from comment #99) > Version: 8.6.0.201502131403 > GNU gdb (GDB) CentOS (7.0.1-45.el5.centos) Could you try things with the latest CDT 8.7/Mars? Also, GDB 7.0 is really old and is not officially supported by CDT. It would be great if you could try with the latest GDB 7.9.
Sorry, I have no root access to the system and GDB 7.9 is not available there. And our IT deliberately keeps archaic software - to not to have work all the dependencies again - and/or to torture us users...
(In reply to vit bernatik from comment #102) > Sorry, I have no root access to the system and GDB 7.9 is not available > there. And our IT deliberately keeps archaic software - to not to have work > all the dependencies again - and/or to torture us users... Btw, you don't need root access to download and compile your own version of gdb.
If you guide me then I can try. So far I tried ./configure make I got: xml-support.c: In function 'gdb_xml_start_element_wrapper': xml-support.c:329: warning: implicit declaration of function 'XML_StopParser' xml-support.c:329: error: 'XML_FALSE' undeclared (first use in this function) xml-support.c:329: error: (Each undeclared identifier is reported only once xml-support.c:329: error: for each function it appears in.) xml-support.c: In function 'gdb_xml_end_element_wrapper': xml-support.c:412: error: 'XML_FALSE' undeclared (first use in this function) xml-support.c: In function 'gdb_xml_fetch_external_entity': xml-support.c:495: error: storage size of 'status' isn't known xml-support.c: In function 'gdb_xml_use_dtd': xml-support.c:544: warning: implicit declaration of function 'XML_UseForeignDTD' xml-support.c:544: error: 'XML_TRUE' undeclared (first use in this function) xml-support.c: In function 'gdb_xml_parse': xml-support.c:561: error: storage size of 'status' isn't known make[2]: *** [xml-support.o] Error 1 make[2]: Leaving directory `/import/home/vbe/0/instal/gdb79/gdb-7.9/gdb' make[1]: *** [all-gdb] Error 2 make[1]: Leaving directory `/import/home/vbe/0/instal/gdb79/gdb-7.9' make: *** [all] Error 2
