Community
Participate
Working Groups
I'm a NOOB trying to do my very first push into http://git.eclipse.org/c/emf/org.eclipse.emf.eson.git, and it took me a moment to understand my correct Git identify at eclipse.org.. To help others with this in the future, I've just written a new section named "Your Git identify at eclipse.org" on https://wiki.eclipse.org/Git#Your_Git_identify_at_eclipse.org which perhaps you'd like to review. In particular the "Eclipse.org site account password as set on https://dev.eclipse.org/site_login/myaccount.php. TODO clarify relationship to and role of https://git.eclipse.org/r/#/settings/http-password" line.
If I may, please let me take this bug to another level. I'd like to get rid of pure Git. Some projects use Git, some use Gerrit, they are different systems and it is confusing for everyone. From a developer's point of view, Gerrit offers everything that Git does and more -- fine-grained access control, code review, direct pushes without review, and more. Gerrit also gives us https access and SSH key management for free. Pure Git only gives us Git. On our servers, Pure Git is scalable and load-balanced across multiple servers, which is a definite advantage and the only blocker to actually making this happen. Gerrit operates on a single server at Eclipse.org, since having multiple Gerrit masters is still complex. The Gerrit server has never failed yet, but it eventually will.
I'd argue that you want to keep both. They serve different purposes in a build life cycle. Gerrit is for Code Review and approval, where the pure git repo is the master repository. The one point of truth. Once something is approved, reviewed, tested, etc. Then Gerrit pushes up to the master repo. It keeps a good separation from work in progress, and one doesn't need to clone and potentially bring down all the branches that may end up in Gerrit that are never needed. I'd hate to loose that separation. I know for Andmore, I'm think of keeping the main repo at GitHub, but use Gerrit and possibly the GitHub/Gerrit integration for code reviews and contributions. That way if a pull request comes in on the GitHub site, it gets sent to Gerrit for the actual review and things are kept in sync. Anyways, I'd prefer keeping the two, or just move all eclipse.org projects to GitHub and let that be the master repo since they already have good social coding tools built in there and hooks to hook into Gerrit if necessary.
> To help others with this in the future, I've just written a new section > named "Your Git identify at eclipse.org" on > https://wiki.eclipse.org/Git#Your_Git_identify_at_eclipse.org which perhaps > you'd like to review. Thank you. I've added: https://wiki.eclipse.org/Git#Git_or_Gerrit.3F and expanded your section.
(In reply to David Carver from comment #2) > I'd argue that you want to keep both. They serve different purposes in a > build life cycle. Gerrit is for Code Review and approval, where the pure > git repo is the master repository. The one point of truth. > > Once something is approved, reviewed, tested, etc. Then Gerrit pushes up > to the master repo. It keeps a good separation from work in progress, and > one doesn't need to clone and potentially bring down all the branches that > may end up in Gerrit that are never needed. When you use Gerrit this separation is done between automatically managed review branches where work in progress is reviewed, (under refs/changes/) and normal git branches like master, maintenance branches, etc. (under refs/heads/) Gerrit review branches are located in a custom ref namespace refs/changes/ and therefore they are not advertised to clients hence they are not fetched during standard fetch or clone, only if you explicitly specify the complete ref you want to fetch
I support the idea of dropping plain Git and make everyone use Gerrit (with or without code-review, as each project would prefer). It wouldn't be any functional loss, and it would make project more consistent, and it's one less thing to maintain by the webmaster and the foundation. However, I wouldn't dropping the plain Git server also remove the web viewer for source? This is something I believe we'd still need in any case, and I don't think Gerrit allows a good way to browse sources.
(In reply to Mickael Istria from comment #5) > I support the idea of dropping plain Git and make everyone use Gerrit (with > or without code-review, as each project would prefer). It wouldn't be any > functional loss, and it would make project more consistent, and it's one > less thing to maintain by the webmaster and the foundation. > However, I wouldn't dropping the plain Git server also remove the web viewer > for source? This is something I believe we'd still need in any case, and I > don't think Gerrit allows a good way to browse sources. Which is why if pure git is dropped, I'd just recommend the use of Github as the repo. Just my opinion.
> It wouldn't be any functional loss This is the key point. > However, I wouldn't dropping the plain Git server also remove the web viewer > for source? Absolutely not. Gerrit is still Git nonetheless, and the cGit browser would stay. But without a dual Git/Gerrit environment, we could invest some time on better integration between cGit with Gerrit.
> Which is why if pure git is dropped, I'd just recommend the use of Github as > the repo. Just my opinion. I don't think this is a direction we want to take. As pragmatic as it may seem, some projects may not be comfortable, or may simply not want to use GitHub.
(In reply to Denis Roy from comment #7) > Absolutely not. Gerrit is still Git nonetheless, and the cGit browser would > stay. But without a dual Git/Gerrit environment, we could invest some time > on better integration between cGit with Gerrit. Ok, so Gerrit + cGit browser sounds perfect IMO.
Another option for browsing git repositories instead of cgit would be gitiles [1][2] which can also be plugged into Gerrit and then also respects Gerrit access controls. You can try it here https://gerrit.googlesource.com/ [1] https://gerrit.googlesource.com/gitiles/+/master/README [2] https://gerrit.googlesource.com/plugins/gitiles/+/299cd923c0b275d821c0faea65a2fb59603ccc64
You can also duplicate the same behaviour we get from our standard pre-receive hook using Gerrit permissions. See bug 418966 comment #14 PW
This has been dragging and I feel for Denis who "dreams that we only use Gerrit for all our SCM needs". What are the issues to get this done? Is the infrastructure support mentioned in comment 1 still poor for Gerrit? The other issue, which seems to be the main one, is simply a question of pushback from the community. So here is what I suggest: Tell everyone that the git URLs are changing and provide them with the new URLs. Don't mention Gerrit at all. Give a date when the old URLs will be shut-off. If the infrastructure behaves, no one will know the difference and no one will complain. Heck, could the current URLs automatically be redirected to go to the Gerrit ones instead? Then no one has to do anything, and it because a decision to be taken by the webmasters only.
(In reply to Marc Khouzam from comment #12) > Heck, could the current URLs automatically be redirected to go to the Gerrit > ones instead? Then no one has to do anything, and it because a decision to > be taken by the webmasters only. "and it *becomes* a decision to be taken by the webmasters only"
(In reply to Marc Khouzam from comment #12) > What are the issues to get this done? Is the infrastructure support > mentioned in comment 1 still poor for Gerrit? I don't think there's much pushback, and I believe that Gerrit does support some form of fault tolerance. I just need more days in a week :) > Heck, could the current URLs automatically be redirected to go to the Gerrit > ones instead? Perhaps the http/s ones, but those cloning using SSH cannot be redirected. There are host key and user public key differences. We'll make this happen, it's only a matter of time. Then I can get back to dreaming about pretty white ponies.
(In reply to Denis Roy from comment #14) > (In reply to Marc Khouzam from comment #12) > > What are the issues to get this done? Is the infrastructure support > > mentioned in comment 1 still poor for Gerrit? > > I don't think there's much pushback, and I believe that Gerrit does support > some form of fault tolerance. I just need more days in a week :) you could add a read-only git daemon serving repos from Gerrit's review site. Read-only slaves can be used to distribute the read load. AFAIK there's no open source multi-master yet. > > Heck, could the current URLs automatically be redirected to go to the Gerrit > > ones instead? > > Perhaps the http/s ones, but those cloning using SSH cannot be redirected. > There are host key and user public key differences. also at the moment ports are different and https passwords aren't coming from LDAP, but I guess this can be fixed > We'll make this happen, it's only a matter of time. Then I can get back to > dreaming about pretty white ponies.
(In reply to Denis Roy from comment #14) > I don't think there's much pushback, and I believe that Gerrit does support > some form of fault tolerance. I just need more days in a week :) > [...] > We'll make this happen, it's only a matter of time. Then I can get back to > dreaming about pretty white ponies. Oh, I didn't mean to put pressure. I had mis-understood that the problem was to get people on-board with the idea; if it is not, then I'm relieved :) I was afraid that some of the reactions after simrel moved to use Gerrit URLs, might have caused hesitation to do this for every project. As we saw with simrel though, it was really a question of understanding that nothing really changes from a user-perspective, except the URLs; the rest is just gravy on top (and probably should only be mentioned as a separate concept to avoid confusion).
(In reply to Denis Roy from comment #14) > We'll make this happen, it's only a matter of time. Then I can get back to > dreaming about pretty white ponies. Did Gerrit become that much faster since we addressed Bug 438897? I remember having to change Code Recommenders Snipmatch's URL from the Gerrit-backend to cgit because you complained about our use of Gerrit on Twitter [1]. [1] <https://twitter.com/droy_eclipse/status/484894527233478656>
(In reply to Andreas Sewe from comment #17) > Did Gerrit become that much faster since we addressed Bug 438897? I remember > having to change Code Recommenders Snipmatch's URL from the Gerrit-backend > to cgit because you complained about our use of Gerrit on Twitter [1]. Nope... but I didn't complain. It's just a matter of using the right tool for the job. Gerrit is not a code browser and it is not optimized as such. cGit is a code browser and we can scale it very easily.
(In reply to Denis Roy from comment #18) > (In reply to Andreas Sewe from comment #17) > > Did Gerrit become that much faster since we addressed Bug 438897? I remember > > having to change Code Recommenders Snipmatch's URL from the Gerrit-backend > > to cgit because you complained about our use of Gerrit on Twitter [1]. > > Nope... but I didn't complain. It's just a matter of using the right tool > for the job. Gerrit is not a code browser and it is not optimized as such. > cGit is a code browser and we can scale it very easily. what I don't like about cGit is that it's pretty slowish. Loading http://git.eclipse.org/c/egit/egit.git/ from Germany takes 3 to 5 sec. Compare that to a similar code browser gitiles which we use with Gerrit, loading https://gerrit.googlesource.com/gerrit/ hosted in Mountain View takes 0.3-0.4 sec
I prefer to keep both. For many fixes it's just much faster for me to simply push to Git than going via Gerrit. Especially since the Gerrit web UI isn't that fast (at least on my machine). Also to keep in mind: currently many committers directly use Git. If all those changes go via Gerrit and trigger builds, then this will put more load on the infrastructure.
(In reply to Dani Megert from comment #20) > I prefer to keep both. For many fixes it's just much faster for me to simply > push to Git than going via Gerrit. Especially since the Gerrit web UI isn't > that fast (at least on my machine). > > Also to keep in mind: currently many committers directly use Git. If all > those changes go via Gerrit and trigger builds, then this will put more load > on the infrastructure. On most projects that are using only Gerrit, committers can push directly to master and bypass the review. They're using Gerrit as a Git server, just bypassing the review.
(In reply to Mickael Istria from comment #21) > (In reply to Dani Megert from comment #20) > > I prefer to keep both. For many fixes it's just much faster for me to simply > > push to Git than going via Gerrit. Especially since the Gerrit web UI isn't > > that fast (at least on my machine). > > > > Also to keep in mind: currently many committers directly use Git. If all > > those changes go via Gerrit and trigger builds, then this will put more load > > on the infrastructure. > > On most projects that are using only Gerrit, committers can push directly to > master and bypass the review. They're using Gerrit as a Git server, just > bypassing the review. OK, good. Does this also allow to do all other operations like e.g. adding tags?
(In reply to Dani Megert from comment #22) > (In reply to Mickael Istria from comment #21) > > (In reply to Dani Megert from comment #20) > > > I prefer to keep both. For many fixes it's just much faster for me to simply > > > push to Git than going via Gerrit. Especially since the Gerrit web UI isn't > > > that fast (at least on my machine). > > > > > > Also to keep in mind: currently many committers directly use Git. If all > > > those changes go via Gerrit and trigger builds, then this will put more load > > > on the infrastructure. > > > > On most projects that are using only Gerrit, committers can push directly to > > master and bypass the review. They're using Gerrit as a Git server, just > > bypassing the review. > > OK, good. Does this also allow to do all other operations like e.g. adding > tags? yes, Gerrit is a full blown git server so you can also push tags
(In reply to Matthias Sohn from comment #23) > > OK, good. Does this also allow to do all other operations like e.g. adding > > tags? > > yes, Gerrit is a full blown git server so you can also push tags Gerrit is based on JGit, which doesn't support git hooks (bug 299315), so our hook scripts that would prevent arbitrary branch creations don't work there. I just tried to push a new top-level branch to ssh://mkeller@git.eclipse.org:29418/jdt/eclipse.jdt.git , and I was not stopped. On the other hand, I could not delete the branch through EGit any more: Repository ssh://mkeller@git.eclipse.org:29418/jdt/eclipse.jdt.git cannot delete references Branch refs/heads/master2: You need 'Push' rights with the 'Force Push' flag set to delete references. User: mkeller Please read the documentation and contact an administrator if you feel the configuration is incorrect I could not modify the access rules so that I could delete the branch again. https://git.eclipse.org/r/#/c/52609/ needed a Code-Review I couldn't set. I've now removed the master2 branch via ssh on build.eclipse.org.
I plan to investigate scaling Gerrit @ Eclipse in Q4. Running a stock SSH on port 22 is getting real old.
(In reply to Markus Keller -- away till Oct 13 from comment #24) > (In reply to Matthias Sohn from comment #23) > > > OK, good. Does this also allow to do all other operations like e.g. adding > > > tags? > > > > yes, Gerrit is a full blown git server so you can also push tags > > Gerrit is based on JGit, which doesn't support git hooks (bug 299315), so > our hook scripts that would prevent arbitrary branch creations don't work > there. > > I just tried to push a new top-level branch to > ssh://mkeller@git.eclipse.org:29418/jdt/eclipse.jdt.git , and I was not > stopped. you can configure that in Gerrit, if you have the permission "Create Reference" on refs/heads/* see https://git.eclipse.org/r/Documentation/access-control.html#category_create > On the other hand, I could not delete the branch through EGit any more: > > Repository ssh://mkeller@git.eclipse.org:29418/jdt/eclipse.jdt.git > > cannot delete references > Branch refs/heads/master2: > You need 'Push' rights with the 'Force Push' > flag set to delete references. > User: mkeller > Please read the documentation and contact an administrator > if you feel the configuration is incorrect > > I could not modify the access rules so that I could delete the branch again. > https://git.eclipse.org/r/#/c/52609/ needed a Code-Review I couldn't set. > > I've now removed the master2 branch via ssh on build.eclipse.org.
Unfortunately gerrit is much harder to use than pure git even if you use it to commit directly. When the simrel repo switched to gerrit, it took me a full day to figure out what knobs I had to twirl in Eclipse to connect to the repo again (key setup). That's with various wikis that were trying to help with the transition. Forcing all projects that are not already using gerrit onto it, is going to cause a lot of hardship.
(In reply to Konstantin Komissarchik from comment #27) > Unfortunately gerrit is much harder to use than pure git even if you use it > to commit directly. I find this is mis-leading. Using gerrit (git access) or pure git is exactly the same. As you mention below, the only difficulty is connecting to the new address, and I would like to believe this can be made simple (better documentation, or redirecting old addresses, etc) > When the simrel repo switched to gerrit, it took me a > full day to figure out what knobs I had to twirl in Eclipse to connect to > the repo again (key setup).
*** Bug 420696 has been marked as a duplicate of this bug. ***
> I find this is mis-leading. Using gerrit (git access) or pure git is exactly > the same. I've updated the summary to reflect exactly that.
So today I pushed a commit using refs/for/master to create a review. But I selected the wrong remote by mistake and used direct git instead of the Gerrit URL. So I ended up creating a branch 'refs/for/master' on the real repository; this prevented anyone on the project to push anything to Gerrit anymore. I eventually figured it out (with some help), and deleted that branch. Only allowing the Gerrit URL would avoid such mistakes. I removed the direct Git URL from my personal repo so this does not happen again.
(In reply to Marc Khouzam from comment #31) This couldn't have happened in Eclipse SDK repos, because we have a Git hook that prevents such accidental branch creations: https://wiki.eclipse.org/Platform-releng/Git_Workflows#Branches_in_our_Platform_Repos You may want to install that hook in your repos.
(In reply to Markus Keller from comment #32) > (In reply to Marc Khouzam from comment #31) > This couldn't have happened in Eclipse SDK repos, because we have a Git hook > that prevents such accidental branch creations: > https://wiki.eclipse.org/Platform-releng/ > Git_Workflows#Branches_in_our_Platform_Repos > You may want to install that hook in your repos. I find that removing a repository URL (plain Git) which doesn't provide anything over another one (Gerrit) is a more elegant solution that putting hacks/hooks here and there to "support" the possible issues that can happen when dealing with 2 similar URLs.
(In reply to Mickael Istria from comment #33) The hook is not a hack. In contrast to comment 31 and comment 24, it works. I'd be happy to move to Gerrit-only if the performance and functional problems are solved.
> I'd be happy to move to Gerrit-only if the performance and functional > problems are solved. Have we articulated what those are? I can file bug reports. FWIW: - changing a remote is not always seamless. This is not something specific to Gerrit - using Hooks to enforce ref namespace permissions and access control is a hack. Gerrit has a very complete and robust ACL mechanism. - jGit performance is probably near that of git, and it has improved dramatically in the last couple of years ... One could benchmark it easily enough I presume.
(In reply to Denis Roy from comment #35) For performance, I don't have anything more than your comment 1. If the scaling issue is resolved, then I guess we're good to go. The functional requirement is that the scenarios covered by the pre-receive hook need to be supported in Gerrit (see Wiki link). Notable things that didn't work the last time I checked: - accidental branch creations/deletions should be blocked (comment 24) - but users should be free to create/delete/non-ff-push topic branches - it still needs to be easy for project leads to create a maintenance branch or to delete a tag that was pushed accidentally - The Eclipse SDK has some common repos (eclipse.platform.common, www.eclipse.org/eclipse/news.git) that need to be accessible for anybody who has commit rights in any Platform, JDT, PDE, or Equinox project. It's hard for me to test how well this works via the Gerrit URL, because I don't have a user account that is not in the eclipse.platform group.
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant. -- The automated Eclipse Genie.
*** This bug has been marked as a duplicate of bug 533786 ***
