Community
Participate
Working Groups
build.eclipse.org was initially set up circa 2006, to allow projects to run builds. Some used cronjobs, some installed CruiseControl, and eventually we installed a single Hudson instance. Today, none of that is in use anymore, but build.e.o still hosts a few things: 1. Nightly Builds via https://build.eclipse.org (the SSL libs are obsolete) 2. MySQL database for EclipseLink and Dash 3. A webdev cron for the Packaging website 4. The download file checksummer 5. A couple of antique Infocenters We (webmasters) may still need a generic machine to run cron jobs (3 & 4) but everything else can go. Dash and EclipseLink can run MySQL in a container, nightly builds can be hosted on download.e.o Let's work to find a new home for what is left on it and repurpose the machine towards our k8s cluster.
(In reply to Denis Roy from comment #0) > 5. A couple of antique Infocenters The infocenters have been moved to the cluster. So nothing infocenter-related should run on build.eclipse.org anymore. projects-storage.eclipse.org currently points to build.eclipse.org as SSH endpoint for SSH/SCP tasks related to the download folders. This service will need to be relocated.
> Today, none of that is in use anymore, but build.e.o still hosts a few > things: > > 1. Nightly Builds via https://build.eclipse.org (the SSL libs are obsolete) These can be stored on download.e.o and/or archive.e.o > 2. MySQL database for EclipseLink and Dash Could this be an app on the cluster? > 3. A webdev cron for the Packaging website Not sure about this one. > 4. The download file checksummer This is actually run on the www nodes
There are some cron jobs that run scripts that update/maintain the Dash ("dashboard") database. We're overdue to apply more rigor to the deployment of these assets. Chris and I have had some very high level discussion about his team taking ownership, but not with any detail. We'll need to sort out how to transition this.
Some things that we still need to take care of (found by Mikael): * build is a proxy jump for OMR project to external slave => migrate OMR to cluster * terminate help.eclipse.org on build * we need a machine to interact with LDAP (e.g. a dev node?) * Derby database (used by Sravan) => ask Sravan and migrate to the cluster if it's still required
The are some project references to build.eclipse.org in the source code: https://github.com/search?q=org%3Aeclipse+build.eclipse.org or search on Google: "build.eclipse.org" site:git.eclipse.org Also searching on Google shows a few build references: "build.eclipse.org" site:ci.eclipse.org E.g. to http://build.eclipse.org:31338/dmg-packager some are very old build logs so aren't a problem - [DEBUG] (f) signerUrl = http://build.eclipse.org:31338/sign some are comments: https://ci.eclipse.org/webtools/view/webtools_CI/job/WTP-CI_master/ Results here: http://build.eclipse.org/webtools/committers/wtp-R3.y.0-I/ https://ci.eclipse.org/oomph/job/build-filtered-installer/lastBuild/console + curl -o eclipse-inst-mac64.1596620025983292488.dmg --write-out '%{http_code}\n' -F sign=true -F source=@eclipse-inst-mac64.tar.gz http://build.eclipse.org:31338/dmg-packager some builds are now breaking: https://ci.eclipse.org/rap/job/rap-head-incubator-clientscripting/2494/console [INFO] Adding repository http://build.eclipse.org/rt/rap/base-platforms/3.1/extra-dependencies [ERROR] Internal error: java.lang.RuntimeException: Failed to load p2 repository with ID 'extra-dependencies-repository' from location http://build.eclipse.org/rt/rap/base-platforms/3.1/extra-dependencies/: Unable to connect to repository http://build.eclipse.org/rt/rap/base-platforms/3.1/extra-dependencies/content.xml: Connection refused (Connection refused) -> [Help 1] org.apache.maven.InternalErrorException: Internal error: java.lang.RuntimeException: Failed to load p2 repository with ID 'extra-dependencies-repository' from location http://build.eclipse.org/rt/rap/base-platforms/3.1/extra-dependencies/
The Linux Tools project builds into download.eclipse.org, but maintenance is required on the directory. At one time, I used to ssh into build.eclipse.org, but then I was forced to use sftp. To move to archive, I have to scp to my local host, scp to archive, and delete the old files from the main downloads/linuxtools directory via sftp. How do I delete files/directories after the change?
(In reply to Frederic Gurr from comment #4) > * Derby database (used by Sravan) > => ask Sravan and migrate to the cluster if it's still required We already moved away from Derby database. You can remove derby service.
We have signing service hosted on build.eclipse.org. This needs to be moved.
Yes, I noticed the signing service dependencies as well.
(In reply to Sravan Kumar Lakkimsetti from comment #8) > We have signing service hosted on build.eclipse.org. This needs to be moved. (In reply to Ed Merks from comment #9) > Yes, I noticed the signing service dependencies as well. Indeed, and this will taken care of early next year.
FTR the Buildship project also uses the signing service on build.eclipse.org.
*** Signing services won't disappear with the removal of build.eclipse.org *** We will publish a new version of the maven signing plugins that will point to the new urls where signing services will be available. From a project point of view, it will be just a matter of using the newest versions. For those of you who don't use the maven signing plugins but the REST API directly, you will need to update the URL. No other change will be required for the signing services.
> To move to archive, I have to scp to my local host, scp to archive, and > delete the old files from the main downloads/linuxtools directory via sftp. > > How do I delete files/directories after the change? If you're logged in to the Eclipse website, navigate to the directory view on download.eclipse.org (it will show you as logged out, which is a bug). From there you have an option to Archive directories and files, where they are moved, preserving the same paths, to archive.eclipse.org. On archive.eclipse.org, if logged in, you should have options to permanently delete files & directories.
My last message was ambiguous. If you're logged in to the Eclipse website, navigate to the directory view on https://download.eclipse.org/path/to/your/project (it will show you as logged out, which is a bug). From there, committers on their own projects have an option to Archive directories and files, where they are moved, preserving the same paths, to archive.eclipse.org. Navigating to https://archive.eclipse.org/path/to/your/project, if logged in, you should have options to permanently delete files & directories.
> http://build.eclipse.org/rt/rap/base-platforms/3.1/extra-dependencies/: > Unable to connect to repository > http://build.eclipse.org/rt/rap/base-platforms/3.1/extra-dependencies/ > content.xml: Connection refused (Connection refused) - Coincidentally, the http server on build.e.o had died, but it was restarted.
We're one step closer to be able to do this with the resolution of bug 551773. Expect some more updates shortly.
I've sent a reminder to cross-project-issues-dev
Question: I am currently using scp to build.eclipse.org to do maintenance for our linuxtools downloads. The web interface is not adequate for this. I need to put files there (like the sources for the various builds) and to rename. How do I do this after build.eclipse.org is gone? I have tried scp'ing into download.eclipse.org and this times out. I have also tried projects-storage.eclipse.org and this also doesn't work.
(In reply to Jeff Johnston from comment #18) > Question: I am currently using scp to build.eclipse.org to do maintenance > for our linuxtools downloads. The web interface is not adequate for this. > I need to put files there (like the sources for the various builds) and to > rename. > > How do I do this after build.eclipse.org is gone? I have tried scp'ing into > download.eclipse.org and this times out. I have also tried > projects-storage.eclipse.org and this also doesn't work. The recommended way is: automate such tasks and run them from your project's Jenkins instance, which has SSH access to download.eclipse.org via projects-storage.eclipse.org.
(In reply to Frederic Gurr from comment #19) > (In reply to Jeff Johnston from comment #18) > > Question: I am currently using scp to build.eclipse.org to do maintenance > > for our linuxtools downloads. The web interface is not adequate for this. > > I need to put files there (like the sources for the various builds) and to > > rename. > > > > How do I do this after build.eclipse.org is gone? I have tried scp'ing into > > download.eclipse.org and this times out. I have also tried > > projects-storage.eclipse.org and this also doesn't work. > The recommended way is: automate such tasks and run them from your project's > Jenkins instance, which has SSH access to download.eclipse.org via > projects-storage.eclipse.org. Unfortunately many of these processes are manually implemented. So, I have to create a jenkins job to copy something to/from a git repo? This really ties my hands in making a milestone or release. Couldn't there be some way to allow scp to some eclipse.org machine with access to the downloads directory?
(In reply to Jeff Johnston from comment #20) > Unfortunately many of these processes are manually implemented. So, I have > to create a jenkins job to copy something to/from a git repo? This really > ties my hands in making a milestone or release. Couldn't there be some way > to allow scp to some eclipse.org machine with access to the downloads > directory? Could there be some set of jobs set up around common tasks (copying/moving, deleting content) ? Tasks like cleaning up project folders ( https://download.eclipse.org/oomph/archive/eclipse/ ) could certainly be common among many projects.
(In reply to Roland Grunberg from comment #21) > Could there be some set of jobs set up around common tasks (copying/moving, > deleting content) ? Tasks like cleaning up project folders ( > https://download.eclipse.org/oomph/archive/eclipse/ ) could certainly be > common among many projects. Since projects are free to structure their downloads directory as they want, it's not easy to create jobs that are universal enough for every project. I'm not sure it's even necessary, since all it takes is a job that has the SSH agent option enabled, the right credentials selected (genie.xxx@projects-storage.eclipse.org) and a shell build step like ssh genie.xxxh@projects-storage.eclipse.org << EOF cp x y mv y z rm rf x EOF This can be improved by using parameterized builds.
FYI, With this I tried to make that a little easier: https://download.eclipse.org/justj/www/download.eclipse.org.php?parambuild=true I.e., actions that launch a parameterized job. I see the description formatting is poor on the new infrastructure...
(In reply to Ed Merks from comment #23) > I see the description formatting is poor on the new infrastructure... If that's the case, it's either a missing configuration or a missing plugin. Please point me to the affected CI instance.
(In reply to Frederic Gurr from comment #24) > (In reply to Ed Merks from comment #23) > > I see the description formatting is poor on the new infrastructure... > If that's the case, it's either a missing configuration or a missing plugin. > Please point me to the affected CI instance. Actually, I think it's always been this way. In the configuration the description for the SSH parameter looks fine in the preview: https://ci.eclipse.org/justj/job/internal/job/genie.justj-ssh/configure And it looks fine when I launch it normally: https://ci.eclipse.org/justj/job/internal/job/genie.justj-ssh/build?delay=0sec But when I launch it like this as a parambuild https://ci.eclipse.org/justj/job/internal/job/genie.justj-ssh/parambuild/?SSH=cd+%2Fhome%2Fdata%2Fhttpd%2Fdownload.eclipse.org%2Fjustj%2F%0Acat+%3E+new_file+%3C%3C%22END_OF_FILE_CONTENT%22%0AEND_OF_FILE_CONTENT%0A The description shows the HTML markup. In any case, this type of job, even without the parambuild to drive it is useful for doing quick shell things.
Brownout schedule, in Eastern times: Tuesday July 13, 9am - 11am Tuesday July 20, 9am - 2pm Thursday July 29, 9am - 9am (24h) Tentative shutdown: Tuesday, Aug. 10.
(In reply to Denis Roy from comment #26) > Brownout schedule, in Eastern times: > > Tuesday July 13, 9am - 11am > Tuesday July 20, 9am - 2pm > Thursday July 29, 9am - 9am (24h) > > Tentative shutdown: Tuesday, Aug. 10. I'm unable to enact these brownouts until the new project-storage VM is in place.
The new project-storage machine is in place and serving clients. We did a brownout yesterday/today and while a few minor things showed up we look to be in good shape. Let's move the shutdown to the end of August with one more brownout & notice to committers to give everyone a chance. -M.
When trying to deploy the Scout M2 contribution the job can't scp to projects-storage.eclipse.org anymore. It hangs: https://ci.eclipse.org/scout/job/org.eclipse.scout.sdk_deploy_from_tag_11/19/console It seems to have stopped working with the move to the new projects-storage machine.
see also https://bugs.eclipse.org/bugs/show_bug.cgi?id=575202
What should have been a short brownout is longer than expected due to the outage. Please see https://www.eclipsestatus.io/incidents/rsr9qzb1yl2h
(In reply to Arthur van Dorp from comment #29) > When trying to deploy the Scout M2 contribution the job can't scp to > projects-storage.eclipse.org anymore. It hangs: > https://ci.eclipse.org/scout/job/org.eclipse.scout.sdk_deploy_from_tag_11/19/ > console > It seems to have stopped working with the move to the new projects-storage > machine. Seems to work again since build #20 (https://ci.eclipse.org/scout/job/org.eclipse.scout.sdk_deploy_from_tag_11/20/console).
We are planning one final brownout - Aug 31, 9am EDT for 24h Final Shutdown planned for September 10th, 9am EDT
(In reply to Denis Roy from comment #33) > We are planning one final brownout - Aug 31, 9am EDT for 24h > > Final Shutdown planned for September 10th, 9am EDT Sounds good Denis. On mailing list you mentioned "After a cool-down period". As the shutdown is currently scheduled to be after the 2021-09 final build date I hope we are done with it fully and all projects are indeed moved off it. Can you make the cool-down period last until the 2021-09 is fully out the door so in case there is a last minute respin we handle it. That date is 15 Sept. Knock on wood that we have no respins needed though! And if we do, that no projects are accidentally still reliant on build.eclipse.org.
(In reply to Jonah Graham from comment #34) > (In reply to Denis Roy from comment #33) > > We are planning one final brownout - Aug 31, 9am EDT for 24h > > > > Final Shutdown planned for September 10th, 9am EDT > > Sounds good Denis. > > On mailing list you mentioned "After a cool-down period". As the shutdown is > currently scheduled to be after the 2021-09 final build date I hope we are > done with it fully and all projects are indeed moved off it. Can you make > the cool-down period last until the 2021-09 is fully out the door so in case > there is a last minute respin we handle it. That date is 15 Sept. That's not a problem at all. We plan on keeping it kicking around for longer before repurposing it.
(In reply to Denis Roy from comment #33) > We are planning one final brownout - Aug 31, 9am EDT for 24h build.eclipse.org is offline for a final 24h brownout.
build is back online. Final Shutdown planned for September 10th, 9am EDT.
Bye bye build: https://blogs.eclipse.org/post/denis-roy/bye-bye-build-end-era
New Gerrit change created: https://git.eclipse.org/r/c/rcptt/org.eclipse.rcptt/+/192650
Gerrit change https://git.eclipse.org/r/c/rcptt/org.eclipse.rcptt/+/192650 was merged to [master]. Commit: http://git.eclipse.org/c/rcptt/org.eclipse.rcptt.git/commit/?id=71c63b21b4028e1de1f68151712361137adf1381
Eclipse Virgo and Gemini projects are still reliant on build.eclipse.org. I am running into several errors when trying to build them. I need to submit a patch to upgrade tomcat. What would be the fix for this?
(In reply to vikram bansal from comment #41) The hardware behind build.e.o was reclaimed some time ago, so those projects will need to update their builds/processes to use the newer alternatives as mentioned in some of the previous comments on this bug. Your best bet for contacting the projects is probably via their associated 'dev' mailing lists(https://projects.eclipse.org/projects/rt.virgo/contact and https://projects.eclipse.org/projects/rt.gemini/contact ). My understanding is that at least some of the Gemini subprojects are currently undergoing termination reviews, so you may not receive any responses.
