Bug 537913 - Investigate foundation-hosted GitLab
Summary: Investigate foundation-hosted GitLab
Status: RESOLVED FIXED
Alias: None
Product: Community
Classification: Eclipse Foundation
Component: Bugzilla (show other bugs)
Version: unspecified   Edit
Hardware: All All
: P3 enhancement (vote)
Target Milestone: ---   Edit
Assignee: Eclipse Webmaster CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 502323 (view as bug list)
Depends on:
Blocks:
 
Reported: 2018-08-14 01:31 EDT by Markus Karg CLA
Modified: 2020-10-19 10:27 EDT (History)
26 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Markus Karg CLA 2018-08-14 01:31:35 EDT
I'd like to propose that the EF runs its own GitLab server farm.

GitLab is a great and widely spread open source tool which mostly resembles what GitHub does. It contains a lot of services, and all the services are greatly working together:

* Issue Tracker (could replace Bugzilla)
* Agile project management
* Git + LFS
* CI / DI included (Gitlab CI Runners could replace Jenkins)
* Can integrate with existing services, in particular can mirror GitHub.

In the future, it could be a great replacement for many other tools currently in place. And for the present it would be a great tool for those projects not wanting to use GitHub (as it is external to the EF).
Comment 1 Denis Roy CLA 2019-07-19 09:28:10 EDT
We likely qualify for free, on-prem Entreprise edition.

https://about.gitlab.com/solutions/open-source/program/
Comment 2 Wayne Beaton CLA 2019-07-24 22:23:17 EDT
*** Bug 502323 has been marked as a duplicate of this bug. ***
Comment 3 Nobody - feel free to take it CLA 2019-08-29 07:45:09 EDT
Thanks to Denis Roy for pointing me here.

Is GitLab used for issue tracking by any project of size comparable to bugs.eclipse.org?

Does GitLab issues allow tracking importance? At first sight, they don't. When I searched for a ticket about that, I found one about priority instead: https://gitlab.com/gitlab-org/gitlab-ce/issues/25508
I only checked quickly, but this doesn't look good. It looks like a ticket could have multiple priority labels at the same time. Labels can be relevant, but trying to do so much with them suggests the product is still far from mature.

As reported in https://gitlab.com/gitlab-org/gitlab-ce/issues/23530 there is currently no migration path from Bugzilla. So I'm afraid this request cannot be fulfilled without a colossal effort involving important development at this time.


That being said, I very much agree that Bugzilla should be dropped as soon as possible. The other candidate I see is Redmine, which is implemented in Ruby and mature, but I don't know it enough to recommend more than an evaluation.
Comment 4 Markus Karg CLA 2019-08-29 08:59:04 EDT
(In reply to Filipus Klutiero from comment #3)
> Thanks to Denis Roy for pointing me here.
> 
> Is GitLab used for issue tracking by any project of size comparable to
> bugs.eclipse.org?
> 
> Does GitLab issues allow tracking importance? At first sight, they don't.
> When I searched for a ticket about that, I found one about priority instead:
> https://gitlab.com/gitlab-org/gitlab-ce/issues/25508
> I only checked quickly, but this doesn't look good. It looks like a ticket
> could have multiple priority labels at the same time. Labels can be
> relevant, but trying to do so much with them suggests the product is still
> far from mature.
> 
> As reported in https://gitlab.com/gitlab-org/gitlab-ce/issues/23530 there is
> currently no migration path from Bugzilla. So I'm afraid this request cannot
> be fulfilled without a colossal effort involving important development at
> this time.
> 
> 
> That being said, I very much agree that Bugzilla should be dropped as soon
> as possible. The other candidate I see is Redmine, which is implemented in
> Ruby and mature, but I don't know it enough to recommend more than an
> evaluation.

See https://about.gitlab.com/customers/. Are Cloud Native Foundation, CERN, ESA, SIEMANS and Goldman Sachs big enough to be comparable? The software itself is also available as cloud-hosted offering, with thousands of projects, so I doubt it will fail at the EF.

If you have ever used Gitlab and Redmine, you won't touch Redmine anymore. It is just crap and way obsoleted by Gitlab.

BTW, there *are* migration paths from Bugzilla, and the needed development is *not* "colossal".
Comment 5 Nobody - feel free to take it CLA 2019-08-31 20:33:18 EDT
(In reply to Markus Karg from comment #4)
> (In reply to Filipus Klutiero from comment #3)
> > Thanks to Denis Roy for pointing me here.
> > 
> > Is GitLab used for issue tracking by any project of size comparable to
> > bugs.eclipse.org?
> > 
> > Does GitLab issues allow tracking importance? At first sight, they don't.
> > When I searched for a ticket about that, I found one about priority instead:
> > https://gitlab.com/gitlab-org/gitlab-ce/issues/25508
> > I only checked quickly, but this doesn't look good. It looks like a ticket
> > could have multiple priority labels at the same time. Labels can be
> > relevant, but trying to do so much with them suggests the product is still
> > far from mature.
> > 
> > As reported in https://gitlab.com/gitlab-org/gitlab-ce/issues/23530 there is
> > currently no migration path from Bugzilla. So I'm afraid this request cannot
> > be fulfilled without a colossal effort involving important development at
> > this time.
> > 
> > 
> > That being said, I very much agree that Bugzilla should be dropped as soon
> > as possible. The other candidate I see is Redmine, which is implemented in
> > Ruby and mature, but I don't know it enough to recommend more than an
> > evaluation.
> 
> See https://about.gitlab.com/customers/. Are Cloud Native Foundation, CERN,
> ESA, SIEMANS and Goldman Sachs big enough to be comparable?


I don't know them all, but assuming you mean SIEMENS, these are organizations, not projects. Even then, I looked at 2 case studies to get an idea, and I don't see a word about issue tracking in ESA's or CERN's. For all I've seen on these pages, these organizations may not have any GitLab ticket.

> The software
> itself is also available as cloud-hosted offering, with thousands of
> projects, so I doubt it will fail at the EF.


But how many tickets does it have?

> If you have ever used Gitlab and Redmine, you won't touch Redmine anymore.
> It is just crap and way obsoleted by Gitlab.


I've used both a bit.
 
> BTW, there *are* migration paths from Bugzilla, and the needed development
> is *not* "colossal".

No one claimed the development needed was colossal, but thanks. If you are referring to bugzilla2gitlab, it would help to see a demonstration. The ticket documents that it features "Mappings to set how Bugzilla components are translated into GitLab labels". In other words, GitLab Issues seems to be so labelist it doesn't have any equivalent to components other than labels. To make this manageable, Diego Giovane Pasqualin requested the ability to group labels of similar purposes: https://gitlab.com/gitlab-org/gitlab-ce/issues/21902
There is both good and bad in this request's handling. It was recently fulfilled with what GitLab calls "scoped labels": https://docs.gitlab.com/ee/user/project/labels.html#scoped-labels-premium
But only in GitLab Enterprise Edition. Nevertheless, the ticket - which was requesting the feature in Community Edition - was unceremoniously closed, which indicates frail QA from GitLab.
Labels are something good, but stretching their use to replace proper fields makes the system inconvenient. bugs.eclipse.org would be much less convenient if components had to be replaced with labels (although they do have one major advantage since they do not require reporters to define the components).

And what about products? What would create all of Eclipse's products? Could tickets mistakenly reported to a certain product be moved to another? The fate of the above ticket suggests no.


In summary, from what I see, GitLab's Issues can be good for small projects, which have a small number of issues. They can be setup quickly, and at some point in growth, the development team can add labels to manage the number of tickets. But they're not designed to handle hundreds of thousands of issues, when statuses, importance, products and components are necessary to manage the volume.
Comment 6 Denis Roy CLA 2019-11-27 14:42:38 EST
https://gitlab-test.eclipse.org/ is up and running, and you can log in using your Eclipse Foundation account.

One noteworthy thing - although we see it as a replacement for Bugzilla and Git (and others), unlike our tightly controlled Bugzilla/Gerrit, GitLab follows the fork model, similar to GitHub.

This means anyone can host a fork'ed repo. In fact, anyone can create their own personal public repos, just like GitHub.

I've restricted everything to "public" since we don't want any private repos.

My intent is to create one GitLab Group per Eclipse Project. The GL group is like a GH organization. I am following a feature request on GitLab, that could prevent users from creating projects outside the Group scope: https://gitlab.com/gitlab-org/gitlab/issues/1112

Please kick the tires and follow-up with your comments.
Comment 7 Gunnar Wagenknecht CLA 2019-11-28 15:28:47 EST
(In reply to Denis Roy from comment #6)
> This means anyone can host a fork'ed repo. In fact, anyone can create their
> own personal public repos, just like GitHub.

Did you see the workaround mentioned on the GitLab issue? I believe this would better serve the Foundation's needs, i.e. only Webmaster will be able to create repos. This is similar to how GitHub integration work, isn't it? I think it's ok to open tickets for requesting new repos - unless you see it as a burden for Webmaster and would like to delegate to projects. :)
Comment 8 Denis Roy CLA 2019-11-29 06:40:41 EST
(In reply to Gunnar Wagenknecht from comment #7)
> (In reply to Denis Roy from comment #6)
> > This means anyone can host a fork'ed repo. In fact, anyone can create their
> > own personal public repos, just like GitHub.
> 
> Did you see the workaround mentioned on the GitLab issue?

Can you be more specific?  I browsed through the comments but didn't see anything.


> I believe this
> would better serve the Foundation's needs, i.e. only Webmaster will be able
> to create repos. This is similar to how GitHub integration work, isn't it? I
> think it's ok to open tickets for requesting new repos - unless you see it
> as a burden for Webmaster and would like to delegate to projects. :)

I'm not sure if allowing just anyone to create a repo at Eclipse is a bad thing. One thing -- if we do entertain the idea of generic repos and official project ones, we'd definitely need a visual way of identifying them.
Comment 9 Gunnar Wagenknecht CLA 2019-11-29 07:36:40 EST
(In reply to Denis Roy from comment #8)
> (In reply to Gunnar Wagenknecht from comment #7)
> > (In reply to Denis Roy from comment #6)
> > > This means anyone can host a fork'ed repo. In fact, anyone can create their
> > > own personal public repos, just like GitHub.
> > 
> > Did you see the workaround mentioned on the GitLab issue?
> 
> Can you be more specific?  I browsed through the comments but didn't see
> anything.

https://gitlab.com/gitlab-org/gitlab/issues/1112#note_16828012

> Right now, you can set the project limit to 0 and users won't be able to 
> create any projects whatsoever. As a current workaround they could use 
> that and have only admins create projects in groups [...]

> I'm not sure if allowing just anyone to create a repo at Eclipse is a bad
> thing. One thing -- if we do entertain the idea of generic repos and
> official project ones, we'd definitely need a visual way of identifying them.

Exactly: it is scope creep. There is a ton of additional needs and responsibilities once you start hosting arbitrary user repositories. I don't think the EF should be in the position of competing with GitHub and other free hosting services for hosting free Git repositories. Just projects. That's it. (IMO)
Comment 10 Denis Roy CLA 2019-11-29 09:05:40 EST
> > > > This means anyone can host a fork'ed repo. In fact, anyone can create their
> > > > own personal public repos, just like GitHub.
> > > 
> > > Did you see the workaround mentioned on the GitLab issue?
> > 
> > Can you be more specific?  I browsed through the comments but didn't see
> > anything.
> 
> https://gitlab.com/gitlab-org/gitlab/issues/1112#note_16828012
> 
> > Right now, you can set the project limit to 0 and users won't be able to 
> > create any projects whatsoever. As a current workaround they could use 
> > that and have only admins create projects in groups [...]

I had that initially set to 0, but it breaks the easy fork-edit-submit model that makes Git[Hub|Lab] so easy to use, and compelling for fly-by contributions.
Comment 11 Denis Roy CLA 2019-11-29 09:12:17 EST
I do believe that this proposal is in-line with what we would want:
https://gitlab.com/gitlab-org/gitlab/issues/1112#note_72515013

"For open-source projects, it would be useful for arbitrary members of the public to be able to post their changes and submit MRs against extant projects, but no more than that."
Comment 12 Gunnar Wagenknecht CLA 2019-11-29 09:48:11 EST
(In reply to Denis Roy from comment #10)
> I had that initially set to 0, but it breaks the easy fork-edit-submit model
> that makes Git[Hub|Lab] so easy to use, and compelling for fly-by
> contributions.

Sight. I hadn't realized that forking requires the same permissions as creating an empty/new.
Comment 13 Andrii Berezovskyi CLA 2019-12-06 15:43:30 EST
Will you consider enabling login through Github? If users need an Eclipse account just to file an issue, it might be a non-started for some projects that want to keep the  barrier as low as possible.
Comment 14 Wim Jongman CLA 2019-12-08 15:14:51 EST
Nice, I love GitLab. We are looking to move WindowBuilder to Github but I would like to explore this route first.
Comment 15 Wim Jongman CLA 2019-12-08 15:32:39 EST
Denis can you add group creation permission to the sandbox?
Comment 16 Denis Roy CLA 2019-12-11 09:07:50 EST
(In reply to Andrii Berezovskyi from comment #13)
> Will you consider enabling login through Github? If users need an Eclipse
> account just to file an issue, it might be a non-started for some projects
> that want to keep the  barrier as low as possible.

Makes sense, I'll add it to the radar. Not sure what that entails but the logic is sound.


> Denis can you add group creation permission to the sandbox?

That is next on the list for the sandbox. In the meanwhile, I've created a repo (project) for WindowBuilder inside a group in which you are the maintainer:

https://gitlab-test.eclipse.org/eclipse-windowbuilder/eclipse-windowbuilder

Please feel free to kick the tires!
Comment 17 Denis Roy CLA 2020-01-08 10:11:08 EST
Overall goals and timeline are on GitLab.com, as the GL team have offered some assistance.

https://gitlab.com/gitlab-org/gitlab/issues/195865
Comment 18 Denis Roy CLA 2020-06-11 14:08:47 EDT
Just a quick status update.

We've set up a production-ready infra in a hosted location in Switzerland.  Gitlab is being set up as we speak. We are targetting the end of the month for new (to Eclipse) projects that do not expect to import/migrate issues & bugs.
Comment 19 Denis Roy CLA 2020-06-19 14:17:31 EDT
https://gitlab.eclipse.org/ is up and running.

I will work with the webdev team to get the ECA validation and team sync mechanisms in place.  We'll soon be ready for early adopter projects who are new to the Eclipse Foundation and do not have issues/bugs trackers to import (unless done manually by the project).
Comment 20 Carsten Reckord CLA 2020-06-19 14:49:59 EDT
Is the plan to also support GitLab CI as a build environment eventually? 

Our in-house experience with both (we're currently in the process of migrating everything over to GitLab CI, after using both in parallel for an extensive time) suggests that it's not just easier for projects to declare readable and clean builds, but the K8s integration is also far better and overall it seems more stable and less resource-hungry than Jenkins. Not sure about scaling it up to EclipseFdn size though...
Comment 21 Mikaël Barbero CLA 2020-06-22 04:15:04 EDT
(In reply to Carsten Reckord from comment #20)
> Is the plan to also support GitLab CI as a build environment eventually? 
> 
> Our in-house experience with both (we're currently in the process of
> migrating everything over to GitLab CI, after using both in parallel for an
> extensive time) suggests that it's not just easier for projects to declare
> readable and clean builds, but the K8s integration is also far better and
> overall it seems more stable and less resource-hungry than Jenkins. Not sure
> about scaling it up to EclipseFdn size though...

We plan to evaluate GitLab CI by the end of the year.
Comment 22 Wim Jongman CLA 2020-06-22 04:41:52 EDT
(In reply to Carsten Reckord from comment #20)
> Is the plan to also support GitLab CI as a build environment eventually? 
> 
> Our in-house experience with both (we're currently in the process of
> migrating everything over to GitLab CI, after using both in parallel for an
> extensive time) suggests that it's not just easier for projects to declare
> readable and clean builds, but the K8s integration is also far better and
> overall it seems more stable and less resource-hungry than Jenkins. Not sure
> about scaling it up to EclipseFdn size though...

Are you comparing Gitlab's internal build system with Jenkins? What does Gitlab CI use as a build engine?
Comment 23 Carsten Reckord CLA 2020-06-22 12:22:47 EDT
(In reply to Mikaël Barbero from comment #21)
> We plan to evaluate GitLab CI by the end of the year.

Okay, let me know if I can help or serve as a lab rat when you do :)

(In reply to Wim Jongman from comment #22)
> Are you comparing Gitlab's internal build system with Jenkins? 

Yes

> What does Gitlab CI use as a build engine?

It's their own engine, in some aspects not unlike TravisCI, but overall I think easier and more intuitive, yet very powerful. It's also very tightly and very neatly integrated into Gitlab workflows, i.e. "just works".

See https://docs.gitlab.com/ee/ci/ for the full documentation, and in particular the .gitlab-ci.yaml reference at https://docs.gitlab.com/ee/ci/yaml/README.html for a good impression of what your typical build might look like.

I can also post an example pipeline if that's of interest.
Comment 24 Carsten Reckord CLA 2020-06-22 12:29:07 EDT
(In reply to Denis Roy from comment #19)
> We'll soon be ready for early adopter projects who are
> new to the Eclipse Foundation and do not have issues/bugs trackers to import
> (unless done manually by the project).

I'd like to volunteer moving MPC over to GitLab. We do have existing issues in Bugzilla, but I'm fine with leaving them there for now. (If there's a decent path for me to migrate manually, I could look into that as well.)

Just one question: What would the contribution workflow for non-committers look like then? Is ECA validation for merge requests already in place, or should we validate manually for now, or stay with Gerrit?
Comment 25 Alberto Debiasi CLA 2020-06-25 04:35:06 EDT
I'd like to migrate the repository of Eclipse CHESS [1] to GitLab.

[1] https://projects.eclipse.org/projects/polarsys.chess
Comment 26 Denis Roy CLA 2020-06-25 10:41:24 EDT
> Just one question: What would the contribution workflow for non-committers
> look like then? Is ECA validation for merge requests already in place, or
> should we validate manually for now, or stay with Gerrit?

Great question! The Webdev team has implemented the same ECA and commit check mechanism that is availble today on Gerrit and Github. The ECA check will be installed today.

No manual checks for you to do.
Comment 27 Denis Roy CLA 2020-06-25 13:40:11 EDT
MPC and Chess should be on GitLab by tomorrow. I'll notify via this bug.

Any other early adopters?
Comment 28 Antonio Garcia-Dominguez CLA 2020-06-26 03:47:26 EDT
I'd like to migrate the repositories of Eclipse Hawk too:

https://projects.eclipse.org/projects/modeling.hawk

If possible, both the website and regular source code repositories.
Comment 29 Joachim Bleidiessel CLA 2020-07-01 10:28:53 EDT
We would also like to migrate our project to the eclipse gitlab instance, if possible:
https://projects.eclipse.org/projects/technology.set

Our project is in the phase of the initial code contribution, therefore we don't have any existing files in our repo and also no bugtracker whatsoever.

Would be good to start our work at gitlab!
Comment 30 Alberto Debiasi CLA 2020-07-06 05:22:43 EDT
(In reply to Denis Roy from comment #27)
> MPC and Chess should be on GitLab by tomorrow. I'll notify via this bug.
> 
> Any other early adopters?

Thanks Denis.

Is the migration of CHESS on GitLab completed?
Comment 31 Denis Roy CLA 2020-07-06 08:39:27 EDT
I'll likely be doing early adopter migrations in about 5 hours!
Comment 32 Denis Roy CLA 2020-07-07 07:47:58 EDT
The CHESS project is migrated (https://gitlab.eclipse.org/eclipse/chess), but please do not use it yet.

We have a small snag with the Project Management Infra (PMI) @ projects.eclipse.org and Gitlab. I'll ping back once it's resolved, then migrate the others in the queue (MPC, Hawk, set).
Comment 33 Denis Roy CLA 2020-07-08 13:19:47 EDT
Thanks for everyone's patience -- we'll have this sorted out very soon.
Comment 34 Denis Roy CLA 2020-07-08 13:41:13 EDT
Hawk is migrated!
https://gitlab.eclipse.org/eclipse/hawk/hawk

Hawk team: please file any Gitlab or migration issues directly on Gitlab:
https://gitlab.eclipse.org/eclipse-foundation/gitlab/-/issues
Comment 35 Denis Roy CLA 2020-07-08 20:37:47 EDT
MPC is migrated!
https://gitlab.eclipse.org/eclipse/mpc/org.eclipse.epp.mpc

MPC team: please file any Gitlab or migration issues directly on Gitlab:
https://gitlab.eclipse.org/eclipse-foundation/gitlab/-/issues
Comment 36 Denis Roy CLA 2020-07-08 20:42:00 EDT
Chess is migrated!
https://gitlab.eclipse.org/eclipse/chess/chess

Chess team: please file any Gitlab or migration issues directly on Gitlab:
https://gitlab.eclipse.org/eclipse-foundation/gitlab/-/issues
Comment 37 Denis Roy CLA 2020-07-08 20:44:08 EDT
(In reply to Joachim Bleidiessel from comment #29)
> We would also like to migrate our project to the eclipse gitlab instance, if
> possible:
> https://projects.eclipse.org/projects/technology.set
> 
> Our project is in the phase of the initial code contribution, therefore we
> don't have any existing files in our repo and also no bugtracker whatsoever.
> 
> Would be good to start our work at gitlab!

Your repo is here:
https://gitlab.eclipse.org/eclipse/set/set

I will update the set project provisioning bug
Comment 38 Denis Roy CLA 2020-07-08 20:47:53 EDT
I'm marking this as FIXED.

All Gitlab-related issues or enhancements should be filed here:
https://gitlab.eclipse.org/eclipsefdn/gitlab/-/issues
Comment 39 Denis Roy CLA 2020-07-09 09:43:41 EDT
I've started a Gitlab quick-start guide, for those of us who are familiar with Bugzilla:

https://wiki.eclipse.org/Gitlab
Comment 40 Dejan Pangercic CLA 2020-10-17 20:13:10 EDT
Hi!

I've read below comments and FAQ: https://www.eclipse.org/europe/faq.php.

We have 1 project candidate that we would like to transfer to https://gitlab.eclipse.org/: https://github.com/eclipse/iceoryx (and after that we would propose some more autonomous driving projects).

I have reviewed projects in https://gitlab.eclipse.org/eclipse and they do not seem to be much alive. 

Are the plans wrt the Eclipse own Gitlab instance more concrete? We find gitlab the best development platform but we would need these features (:
1. CI/CD
2. Container registry (which I see is disabled on gitlab.eclipse.org: https://bugs.eclipse.org/bugs/show_bug.cgi?id=565468)
3. Pages
4. Code quality, code testing and coverage
5. Merge trains

---

If the plans with the self-hosted https://gitlab.eclipse.org aren't ready yet, is it possible to host an eclipse project on gitlab.com?
Comment 41 Denis Roy CLA 2020-10-19 10:27:32 EDT
> We have 1 project candidate that we would like to transfer to
> https://gitlab.eclipse.org/: https://github.com/eclipse/iceoryx (and after
> that we would propose some more autonomous driving projects).

That is great!

> 
> I have reviewed projects in https://gitlab.eclipse.org/eclipse and they do
> not seem to be much alive. 

We're still in early stages and have not publicized the availability much.


> Are the plans wrt the Eclipse own Gitlab instance more concrete?

Everything is always a Work In Progress and subject to improvement.  As it is, the instance is concrete.

> We find
> gitlab the best development platform but we would need these features (:
> 1. CI/CD

We will not be opening up CI/CD at this time until we fully understand how it differentiates from our in-house Jenkins instance (Jiro) at https://ci.eclipse.org/

Unless I'm not understanding correctly, our Jiro is more expandable and flexible than many other offers.


> 2. Container registry (which I see is disabled on gitlab.eclipse.org:
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=565468)

The best way to get that functionality enabled would be to reopen the bug and state your needs :)

> 3. Pages

We are tracking that here:
https://gitlab.eclipse.org/eclipsefdn/gitlab/-/issues/2

> 4. Code quality, code testing and coverage

Jenkins CI at https://ci.eclipse.org can integrate with SonarCloud and possibly other tools.

> 5. Merge trains

I'm not sure I know what that is.

> If the plans with the self-hosted https://gitlab.eclipse.org aren't ready
> yet, is it possible to host an eclipse project on gitlab.com?

That is not a possibility. We think the best options are to work with you, the project owners, in getting gitlab.eclipse.org as functional as possible.