Bug 413527 - [C++14] Implement generalized lambda captures (init-capture)
Summary: [C++14] Implement generalized lambda captures (init-capture)
Status: RESOLVED FIXED
Alias: None
Product: CDT
Classification: Tools
Component: cdt-parser (show other bugs)
Version: 8.2.1   Edit
Hardware: PC Windows 7
: P3 normal with 16 votes (vote)
Target Milestone: 9.5.0   Edit
Assignee: Hansruedi Patzen CLA
QA Contact: Markus Schorn CLA
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 451086
  Show dependency tree
 
Reported: 2013-07-23 09:47 EDT by Joseph Link CLA
Modified: 2018-05-29 11:00 EDT (History)
12 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Joseph Link CLA 2013-07-23 09:47:29 EDT
C++14 will include support for generalized lambda captures described:

http://isocpp.org/files/papers/N3648.html

The following code has been supported since GCC 4.5 and will become standard for C++14:

#include <iostream>
#include <memory>

int main(int argc, char **argv)
{
    std::unique_ptr<int> a(new int(3));

    // declare 'a' and 'b' within the lambda
    // as the deduced type of the initializer
    [a = std::move(a), b = 9]
    {
         std::cout << *a << ' ' << b << std::endl;
    }();

    return 0;
}

CDT 8.2.1 shows the entire lambda as a Syntax Error when this is done.

Given its existing support in GCC and now being on the standards track, it would be nice to take advantage of this as soon as possible,
Comment 1 Francois Ee CLA 2017-03-28 06:01:49 EDT
Is this a duplicate of bug #488265 ? With Neon.2 (4.6.2) I can still see "Syntax Error" for a lambda with captured variables declared.
Comment 2 Nathan Ridge CLA 2017-03-28 14:52:39 EDT
(In reply to Francois Ee from comment #1)
> Is this a duplicate of bug #488265 ?

No, that bug is unrelated. That bug concerns using regular C++11 lambdas in a particular context (inside an initializer list).

This bug concerns a C++14 extension to lambdas that has not been implemented yet.
Comment 3 Patrck McMichael CLA 2018-05-08 11:05:50 EDT
Is anyone working on this bug? It's been over a year since the last update here and close to five years since it was created.  This is pretty close to the most annoying thing about the CDT to me right now. Can anyone point me where to look at it if nobody is working on it now?  How hard of a task is this to someone who can code but is not familiar with eclipse/CDT internals?
Comment 4 Nathan Ridge CLA 2018-05-08 11:37:43 EDT
(In reply to Patrck McMichael from comment #3)
> Is anyone working on this bug?

Not that I'm aware of.

> Can anyone point me
> where to look at it if nobody is working on it now?

I would be happy to provide guidance to someone interested in contributing a fix for this issue!

> How hard of a task is
> this to someone who can code but is not familiar with eclipse/CDT internals?

There are two parts to this fix:

 1) Syntax support. This will avoid init-captures
    causing the entire lambda to be a syntax error.

 2) Semantic support. This will ensure that uses
    of the introduced names inside the lambda
    resolve (otherwise, with only (1) in place,
    there will be localized semantic errors under
    those uses).

I expect (1) to be fairly straightforward. (2) is likely to be more involved, though probably still tractable for a determined contributor.

Let me know if you'd like to give (1) a try, and I'll provide some specific pointers to the relevant code.
Comment 5 Avi Kivity CLA 2018-05-08 11:54:47 EDT
Very poor support for (1) is available in [1]. If the name of the captured variable matches a name the surrounding scope with the same type, it happens to work fairly well (e.g. [x = std::move(x)]). I have used this for several years now.



[1] https://github.com/avikivity/cdt/commit/a7b52ad7019e4c229f21a44ef8fdae2f283da225
Comment 6 Nathan Ridge CLA 2018-05-08 18:09:20 EDT
Thanks, Avi, the patch you posted is a great start as it already eliminates the syntax error.

The next step would be to actually store the initializer expression in the AST instead of just having the parser throw it away after parsing it (this involves modifying the AST node representing captures (CPPASTCapture) to optionally store an initializer expression, and setting the initializer expression in the parser). This would e.g. allow the initializer expression to be syntax-colored.

You are also correct to point out that for the common case of "foo = std::move(foo)", name resolution just works without further changes because the uses inside the lambda just resolve to the variable in the enclosing scope.

(Avi, is there anything else you've been sitting on that you could share with us?  ;))
Comment 7 Avi Kivity CLA 2018-05-08 23:23:04 EDT
I also had a similarly-poor implementation of digit separators, but it was obsoleted by 7d9e0b0ddd7e515966b01c3c7d222ecb71580ff1.
Comment 8 Patrck McMichael CLA 2018-05-09 13:07:07 EDT
Thanks for the updates and pointers, everyone!  Would it be worthwhile to integrate Avi's patch to ease the visually jarring portion of the problem (marking the entire lambda as being syntactically invalid)?

Again, I appreciate the feedback and will see if I can carve out some time to look at this at home (no promises!).
Comment 9 Avi Kivity CLA 2018-05-09 13:14:34 EDT
I don't think the patch is good enough (no unit tests etc.). In case it is, I'll be happy to sign the CLA.

Patrck, in case you follow the dubious practice of installing random software provided by strangers, you can point your Eclipse at http://eclipse.scylladb.com/cdt/ and get a recent cdt.git + this patch. I updated it every few weeks.
Comment 10 Nathan Ridge CLA 2018-05-09 13:19:01 EDT
(In reply to Avi Kivity from comment #9)
> I don't think the patch is good enough (no unit tests etc.). In case it is,
> I'll be happy to sign the CLA.

I would merge this patch with a comment added that says "TODO: store the initializer expression in the AST", and a single unit test that contains a trivial use of an init-capture and checks that there are no syntax errors.

Would you be willing to make those small changes and submit it via Gerrit?
Comment 11 Avi Kivity CLA 2018-05-09 13:37:36 EDT
Certainly. Can you point me to the entry point of the CLA process?
Comment 12 Nathan Ridge CLA 2018-05-09 13:48:51 EDT
(In reply to Avi Kivity from comment #11)
> Certainly. Can you point me to the entry point of the CLA process?

Go to https://accounts.eclipse.org/, sign in, and click on "Eclipse Contributor Agreement" at the top.
Comment 13 Avi Kivity CLA 2018-05-09 13:51:20 EDT
I managed to find it and signed. So I'll submit the patch for review it soon.
Comment 14 Nathan Ridge CLA 2018-05-20 12:52:00 EDT
Just a heads up, in case people would like to have the syntax support in Photon: our photon RC1 build is tomorrow, and RC2 a week after.

I would accept a minimal patch along the lines described in comment 10, before RC2. After that, I would only accept it after Photon branches (in which case the change will not make it into the Photon release).
Comment 15 Avi Kivity CLA 2018-05-20 13:11:45 EDT
I'd love to send a patch, but I can't get "mvn test" to run. I get 

[ERROR] Failed to execute goal on project org.eclipse.cdt.meson: Could not resolve dependencies for project org.eclipse.cdt:org.eclipse.cdt.meson:eclipse-feature:9.5.0-SNAPSHOT: Could not find artifact org.eclipse.cdt:org.eclipse.cdt.meson.docs:jar:1.0.0-SNAPSHOT


Help would be appreciated.
Comment 16 Nathan Ridge CLA 2018-05-20 13:49:15 EDT
I don't know about that particular error, but I don't find the need to run "mvn" commands at all.

To test my changes locally, I just do Run As -> Eclipse Application from within the development instance. If I'm happy with the behaviour, I go ahead and push the patch to gerrit, which will trigger automated tests. If I want to run a particular test suite locally, say ParserTestSuite, I do that with Run As -> JUnit Plug-in Test.
Comment 17 Patrck McMichael CLA 2018-05-24 22:04:43 EDT
Dear Avi Kivity,

Pretty please!

:-)
Comment 18 Avi Kivity CLA 2018-05-26 04:20:55 EDT
(In reply to Nathan Ridge from comment #16)
> I don't know about that particular error, but I don't find the need to run
> "mvn" commands at all.
> 
> To test my changes locally, I just do Run As -> Eclipse Application from
> within the development instance. If I'm happy with the behaviour, I go ahead
> and push the patch to gerrit, which will trigger automated tests. If I want
> to run a particular test suite locally, say ParserTestSuite, I do that with
> Run As -> JUnit Plug-in Test.

That helped a lot! Will talk to Gerrit now.
Comment 19 Avi Kivity CLA 2018-05-26 04:30:32 EDT
I think GDPR killed Gerrit. When I try to sign in, I get complaints that cookies are needed. I am logged in to www.eclipse.org but not to git.eclispe.org.

Meanwhile, a patch with a unit test can be seen in [1].

[1] https://github.com/avikivity/cdt/commit/0686f2c9b0b82f6dd6acae6f0f228663a444b864
Comment 20 Avi Kivity CLA 2018-05-26 04:34:59 EDT
I'm such a n00b. There was a slide-up button in the lower left that allows me to accept cookies.
Comment 21 Avi Kivity CLA 2018-05-26 04:56:04 EDT
Having trouble fetching the commit-msg hook for Change-Id: tag, the instructions are geared towards people with ssh access (I have only http access).
Comment 22 Eclipse Genie CLA 2018-05-26 05:13:44 EDT
New Gerrit change created: https://git.eclipse.org/r/123379
Comment 23 Eclipse Genie CLA 2018-05-26 08:18:27 EDT
New Gerrit change created: https://git.eclipse.org/r/123380
Comment 24 Nathan Ridge CLA 2018-05-26 19:42:17 EDT
Looks like we got not just one patch but two :)

The first patch, by Avi Kivity, takes the minimal approach described in comment 10.

The second patch, by Hansruedi Patzen, implements both of the parts (syntax and semantic) described in comment 4.

Since the second patch involves API changes, and I don't believe we can make API changes to Photon at this stage, what I'll do is merge the first patch into Photon, and merge the second patch (rebased) after Photon branches.
Comment 25 Nathan Ridge CLA 2018-05-26 20:11:53 EDT
(In reply to Nathan Ridge from comment #24)
> Since the second patch involves API changes, and I don't believe we can make
> API changes to Photon at this stage, what I'll do is merge the first patch
> into Photon, and merge the second patch (rebased) after Photon branches.

Looking at Hansruedi's patch made me realize that thre are actually three syntactic forms of an init-capture:

  [foo=expr]
  [foo{expr}]
  [foo(expr)]

Avi's patch gets the parser to accept the first, but not the other two.

We could ask Avi to modify his patch to accept the second two forms, but perhaps a better approach is to ask for an exception to the rule about API changes, and just merge Hansruedi's patch for Photon.

I've posted about this on the CDT mailing list: https://dev.eclipse.org/mhonarc/lists/cdt-dev/msg32954.html
Comment 27 Nathan Ridge CLA 2018-05-28 18:38:15 EDT
(In reply to Nathan Ridge from comment #25)
> perhaps a better approach is to ask for an exception to the rule about API
> changes, and just merge Hansruedi's patch for Photon.
> 
> I've posted about this on the CDT mailing list:
> https://dev.eclipse.org/mhonarc/lists/cdt-dev/msg32954.html

There were no objections to merging Hansruedi's patch into Photon, so I go ahead and did that. It will even make the RC2 build, which has been delayed until tomorrow morning.

So, this is officially fixed for Photon :) Thanks very much to Hansruedi and everyone else who helped move this forward!
Comment 28 Patrck McMichael CLA 2018-05-29 11:00:44 EDT
Wow, great job everyone!  Thank you for proving once again why the CDT is the best C++ IDE out there!