Bug 568936 - Remove remaining CVS defined bundles from Orbit recommended builds starting in 2022-06
Summary: Remove remaining CVS defined bundles from Orbit recommended builds starting i...
Status: RESOLVED FIXED
Alias: None
Product: Orbit
Classification: Tools
Component: bundles (show other bugs)
Version: unspecified   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: 2022-06   Edit
Assignee: Jonah Graham CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 437490 579584 (view as bug list)
Depends on:
Blocks: 499207
  Show dependency tree
 
Reported: 2020-11-18 14:30 EST by Jonah Graham CLA
Modified: 2022-04-27 09:24 EDT (History)
8 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jonah Graham CLA 2020-11-18 14:30:10 EST
There are a few bundles left in CVS that are not in the git repo. 

(In reply to Alexander Kurtakov from comment #6)
> Also looking at the list at least ant, javaewah, javax.annotation, derby,
> httpclient, mockito have newer versions in recipes. Swtchart is eclipse
> project now.
> I haver gone through the list and only the following don't have newer
> version in Orbit:
> * google.inject
> * google.inject.multibinding
> * javax.servlet.jsp
> * olingo
> * tika.parsers
> * snakeyaml (already done)
> * restlet.ext.servlet

This bug is to finish the migration to git and allow orbit to ditch the old CVS repo.
Comment 1 Jonah Graham CLA 2020-11-18 14:45:30 EST
Timing wise I recommend we release 2020-12 with the recently resigned CVS entries. This is because while some of the bundles in CVS have replacements in Git, some of those replacements are actually different bundle names. 

So I will try to create recipes for the bundles (and sets of bundles) in Comment 0 and aim to get them merged by 2021-03 M1.

(Target Milestone to be updated once 2021-03 milestones are added).
Comment 2 Jonah Graham CLA 2020-11-18 16:37:24 EST
*** Bug 437490 has been marked as a duplicate of this bug. ***
Comment 3 Jonah Graham CLA 2020-11-25 11:43:55 EST
Completing this will enable the practice of adding a compare-and-replace option to ensure that all of what Orbit publishes has a consistent signature and qualifier. Currently bundles are re-signed meaning that they are not the same binary content but have the same qualifiers.

Note that the compare and replace can't be done just against the last release (As it may be done for other projects) but has to be done against previous I-build too as many projects (e.g. WTP) consume and then republish the I-build.
Comment 4 Alexander Kurtakov CLA 2020-11-25 11:49:12 EST
(In reply to Jonah Graham from comment #3)
> Completing this will enable the practice of adding a compare-and-replace
> option to ensure that all of what Orbit publishes has a consistent signature
> and qualifier. Currently bundles are re-signed meaning that they are not the
> same binary content but have the same qualifiers.
> 
> Note that the compare and replace can't be done just against the last
> release (As it may be done for other projects) but has to be done against
> previous I-build too as many projects (e.g. WTP) consume and then republish
> the I-build.

I don't think old Orbit releases should be changed. Rather projects should update to latest Orbit. And we should look for a way contributions of old bundles to break contribution to simrel.
Comment 5 Jonah Graham CLA 2020-11-25 11:53:14 EST
(In reply to Alexander Kurtakov from comment #4)
> I don't think old Orbit releases should be changed. 

I am not sure what ^^^ is in reference to?

> Rather projects should
> update to latest Orbit. And we should look for a way contributions of old
> bundles to break contribution to simrel.

That would be nice. Do we have any enabling technology for that today?
Comment 6 Jonah Graham CLA 2020-11-26 15:12:10 EST
As of 2020-12 M3 the following bundles in Orbit are use in simrel and come from CVS definitions:

75 non-source bundles:
ca.odell.glazedlists_1.9.0.v201303080712
com.google.gerrit.common_2.1.5.v201112241444
com.google.gerrit.prettify_2.1.5.v201112241444
com.google.gerrit.reviewdb_2.1.5.v201112241444
com.google.gwtjsonrpc_1.2.5.v201112241444
com.google.gwtorm_1.1.4.v201112241444
com.google.inject_3.0.0.v201605172100
com.google.inject.assistedinject_3.0.0.v201402270930
com.google.inject.multibindings_3.0.0.v201605172100
com.google.protobuf_2.4.0.v201105131100
com.jcraft.jzlib_1.1.1.v201205102305
com.sun.el_2.2.0.v201303151357
java_cup.runtime_0.10.0.v201005080400
javassist_3.13.0.GA_v201209210905
javax.activation_1.1.0.v201211130549
javax.el_2.2.0.v201303151357
javax.inject_1.0.0.v20091030
javax.jws_2.0.0.v201005080400
javax.mail_1.4.0.v201005080615
javax.servlet_3.1.0.v201410161800
javax.servlet.jsp_2.2.0.v201112011158
javax.websocket_1.0.0.v20140310-1603
javax.wsdl_1.6.2.v201012040545
javax.xml_1.3.4.v201005080400
javax.xml.bind_2.2.0.v201105210648
javax.xml.rpc_1.1.0.v201209140446
javax.xml.soap_1.2.0.v201005080501
javax.xml.stream_1.0.1.v201004272200
javax.xml.ws_2.1.0.v200902101523
lpg.runtime.java_2.0.17.v201004271640
net.sourceforge.lpg.lpgjavaruntime_1.1.0.v201004271650
org.antlr.runtime_3.2.0.v201101311130
org.aopalliance_1.0.0.v201105210816
org.apache.axis_1.4.0.v201411182030
org.apache.bcel_5.2.0.v201005080400
org.apache.commons.beanutils_1.8.0.v201205091237
org.apache.commons.cli_1.2.0.v201404270220
org.apache.commons.collections_3.2.2.v201511171945
org.apache.commons.discovery_0.2.0.v201004190315
org.apache.commons.exec_1.1.0.v201301240602
org.apache.commons.httpclient_3.1.0.v201012070820
org.apache.commons.jxpath_1.3.0.v200911051830
org.apache.commons.lang_2.6.0.v201404270220
org.apache.commons.lang3_3.1.0.v201403281430
org.apache.commons.net_3.2.0.v201305141515
org.apache.jasper.glassfish_2.2.2.v201501141630
org.apache.log4j_1.2.15.v201012070815
org.apache.oro_2.0.8.v201005080400
org.apache.poi_3.9.0.v201405241750
org.apache.poi.ooxml_3.9.0.v201405241905
org.apache.poi.ooxml.schemas_3.9.0.v201405241750
org.apache.velocity_1.5.0.v200905192330
org.apache.xerces_2.9.0.v201101211617
org.apache.xml.resolver_1.2.0.v201005080400
org.apache.xml.serializer_2.7.1.v201005080400
org.apache.xmlbeans_2.3.0.v201605172150
org.apache.xmlrpc_3.0.0.v20100427-1100
org.cyberneko.html_1.9.14.v201105210654
org.h2_1.3.168.v201212121212
org.jdom_1.1.1.v201101151400
org.json_1.0.0.v201011060100
org.mozilla.javascript_1.7.5.v201504281450
org.pushingpixels.trident_1.2.0.v201305152020
org.sat4j.core_2.3.5.v201308161310
org.sat4j.pb_2.3.5.v201404071733
org.slf4j.jcl_1.7.2.v20130115-1340
org.swtchart_0.10.0.v201605200358
org.uddi4j_2.0.5.v200805270300
org.w3c.css.sac_1.3.1.v200903091627
org.w3c.dom.events_3.0.0.draft20060413_v201105210656
org.w3c.dom.smil_1.0.0.v200806040011
org.w3c.dom.smil_1.0.1.v200903091627
org.w3c.dom.svg_1.1.0.v201011041433
org.xbill.dns_2.0.8.v201112050911
osgi.enterprise_4.2.0.v201108120515


In addition the following bundles are in Simrel and most of them came from older orbits:

64 non-source bundles:
ch.ethz.iks.r_osgi.remote_1.2.203.v20201026-0028
ch.ethz.iks.r_osgi.remote.source_1.2.203.v20201026-0028
ch.ethz.iks.slp_1.4.0.v20201027-0408
ch.ethz.iks.slp.source_1.4.0.v20201027-0408
com.google.guava_21.0.0.v20170206-1425
com.google.gwt.servlet_2.0.4.v20100709-0658
com.mycorp.examples.timeservice_1.1.200.v20200611-1508
com.mycorp.examples.timeservice.async_2.1.100.v20200611-1508
com.mycorp.examples.timeservice.consumer_1.1.200.v20200611-1508
com.mycorp.examples.timeservice.consumer.ds_1.0.300.v20200611-1508
com.mycorp.examples.timeservice.consumer.ds.async_1.0.200.v20200611-1508
com.mycorp.examples.timeservice.consumer.filediscovery_1.2.101.v20191012-1926
com.mycorp.examples.timeservice.consumer.filediscovery.rest_1.2.101.v20191012-1926
com.mycorp.examples.timeservice.host_1.1.300.v20200611-1508
javax.persistence_2.2.1.v201807122140
javax.servlet_3.0.0.v201112011016
org.apache.batik.bridge_1.6.0.v201011041432
org.apache.batik.bridge_1.7.0.v201011041433
org.apache.batik.bridge.source_1.6.0.v201011041432
org.apache.batik.css_1.6.0.v201011041432
org.apache.batik.css_1.7.0.v201011041433
org.apache.batik.css_1.8.0.v20170214-1941
org.apache.batik.css.source_1.6.0.v201011041432
org.apache.batik.dom_1.6.1.v201505192100
org.apache.batik.dom_1.7.1.v201505191845
org.apache.batik.dom.source_1.6.1.v201505192100
org.apache.batik.dom.svg_1.6.0.v201011041432
org.apache.batik.dom.svg_1.7.0.v201011041433
org.apache.batik.dom.svg.source_1.6.0.v201011041432
org.apache.batik.ext.awt_1.6.0.v201011041432
org.apache.batik.ext.awt_1.7.0.v201011041433
org.apache.batik.ext.awt.source_1.6.0.v201011041432
org.apache.batik.parser_1.6.0.v201011041432
org.apache.batik.parser_1.7.0.v201011041433
org.apache.batik.parser.source_1.6.0.v201011041432
org.apache.batik.pdf_1.6.0.v201105071520
org.apache.batik.svggen_1.6.0.v201011041432
org.apache.batik.svggen_1.7.0.v201011041433
org.apache.batik.svggen.source_1.6.0.v201011041432
org.apache.batik.transcoder_1.6.0.v201011041432
org.apache.batik.transcoder_1.7.0.v201011041433
org.apache.batik.transcoder.source_1.6.0.v201011041432
org.apache.batik.util_1.6.0.v201011041432
org.apache.batik.util_1.7.0.v201011041433
org.apache.batik.util_1.8.0.v20170214-1941
org.apache.batik.util.gui_1.6.0.v201011041432
org.apache.batik.util.gui_1.7.0.v200903091627
org.apache.batik.util.gui_1.8.0.v20170214-1941
org.apache.batik.util.gui.source_1.6.0.v201011041432
org.apache.batik.util.source_1.6.0.v201011041432
org.apache.batik.xml_1.6.0.v201011041432
org.apache.batik.xml_1.7.0.v201011041433
org.apache.batik.xml.source_1.6.0.v201011041432
org.apache.commons.codec_1.13.0.v20200108-0001
org.apache.commons.codec_1.9.0.v20170208-1614
org.apache.hadoop.zookeeper_3.3.3.v201105210832
org.apache.hadoop.zookeeper.source_3.3.3.v201105210832
org.apache.lucene.analyzers-common_6.1.0.v20161115-1612
org.apache.lucene.core_6.1.0.v20170814-1820
org.apache.lucene.queryparser_6.1.0.v20161115-1612
org.gradle.toolingapi_6.3.0.v20200326-1743
org.jivesoftware.smack_3.3.300.v20180303-0057
org.sonatype.m2e.mavenarchiver_0.17.5.202002191804
org.sonatype.m2e.mavenarchiver.feature_0.17.5.202002191804
Comment 7 Jonah Graham CLA 2022-02-22 22:29:41 EST
After discussing with various stakeholders, including at the Planning Council[1] the current plan is to instead of migrating all these bundles is to just deprecate the CVS versions.

Projects consuming the CVS versions of the bundles can point to the last release containing them (will be 2022-03 recommended build*) and/or update the git repo to include them, or better yet consume directly from Maven Central.

Therefore the plan is from 2022-06 M1 the CVS component of this Orbit repo will be dropped.


[1] https://wiki.eclipse.org/Planning_Council/2022-02-02
* please recall that "R" builds in Orbit means recommend, not release.
Comment 8 Alexander Kurtakov CLA 2022-02-23 02:05:09 EST
Go for it - many of these artifacts are vulnerable and there is no easy way to remove them one by one. Combined with the fact there are newer versions in git for a lot of them the only logical step is dropping them altogether.
Comment 9 Lars Vogel CLA 2022-02-24 06:04:52 EST
(In reply to Alexander Kurtakov from comment #8)
> Go for it - many of these artifacts are vulnerable and there is no easy way
> to remove them one by one. Combined with the fact there are newer versions
> in git for a lot of them the only logical step is dropping them altogether.

+1, also we have the direct Maven integration into PDE via m2e these days which is way easier for clients to use.
Comment 10 Eclipse Genie CLA 2022-04-01 09:04:31 EDT
New Gerrit change created: https://git.eclipse.org/r/c/orbit/orbit-recipes/+/192419
Comment 11 Eclipse Genie CLA 2022-04-01 09:20:00 EDT
New Gerrit change created: https://git.eclipse.org/r/c/orbit/orbit-recipes/+/192420
Comment 12 Eclipse Genie CLA 2022-04-01 13:41:16 EDT
New Gerrit change created: https://git.eclipse.org/r/c/orbit/orbit-recipes/+/192428
Comment 15 Christian Dietrich CLA 2022-04-04 10:38:41 EDT
so that idea is that clients of orbit in furture do a


	<location includeAllPlatforms="false" includeConfigurePhase="false" includeMode="planner" includeSource="true" type="InstallableUnit">
		<unit id="x1" version="y1"/>
		<repository location="https://download.eclipse.org/tools/orbit/downloads/x1"/>
	</location>
 ....


	<location includeAllPlatforms="false" includeConfigurePhase="false" includeMode="planner" includeSource="true" type="InstallableUnit">
		<unit id="xn" version="yn"/>
		<repository location="https://download.eclipse.org/tools/orbit/downloads/xn"/>

to me this makes working with orbit a big ton more compilated.
what was the reasoning for dropping all stuff that was there on orbit for a long time?

do we have to readd everything? e.g. google injects needs javax inject
so haveing google inject oin 2022-06 but not having javax.inject sounds pointless
Comment 16 Christian Dietrich CLA 2022-04-04 10:53:20 EDT
is there any tooling to find out what on
2022-06 is now orphaned and unusable cause upstream deps are no longer on orbit
e.g.
org.aopalliance is in the deps of guice 5.0.1 (same as javax.inject)
Comment 17 Jonah Graham CLA 2022-04-04 11:51:08 EDT
(In reply to Christian Dietrich from comment #15)
> so that idea is that clients of orbit in furture do a

Yes and no. Only if they want unmaintained Orbit bundles. This has always been the case and that hasn't changed.

For example, if you want org.objectweb.asm 8.0.1 you need to use 2021-06 or earlier Orbit. If you are happy with org.objectweb.asm 9.x you can use latest Orbit.

The change here is that lots of Orbit bundles (The CVS defined ones) have moved to unmaintained at once. But the reality is that they have been unmaintained for a while, we are just making it official now.
Comment 18 Eclipse Genie CLA 2022-04-04 15:29:59 EDT
New Gerrit change created: https://git.eclipse.org/r/c/orbit/orbit-recipes/+/192472
Comment 19 Eclipse Genie CLA 2022-04-04 15:35:12 EDT
New Gerrit change created: https://git.eclipse.org/r/c/orbit/orbit-recipes/+/192473
Comment 24 Eclipse Genie CLA 2022-04-04 21:31:31 EDT
New Gerrit change created: https://git.eclipse.org/r/c/orbit/orbit-recipes/+/192486
Comment 25 Jonah Graham CLA 2022-04-05 08:10:39 EDT
*** Bug 579584 has been marked as a duplicate of this bug. ***
Comment 26 Jonah Graham CLA 2022-04-05 08:13:37 EDT
*** Bug 579583 has been marked as a duplicate of this bug. ***
Comment 27 Lars Vogel CLA 2022-04-05 08:29:37 EDT
Jonah, creating / updating one configuration per library seems very complex. Did you try to use the Maven integration in to the PDE target platform? This would also allow to create Orbit bundles which the added OSGi metadata.

You would end up with one target definition file defining the libraries and a Tycho build job creating an update site with the feature which can also be defined in the target definition file. Also the target file allows to specify the bdn rules for the conversion of the library.
Comment 28 Jonah Graham CLA 2022-04-05 08:43:46 EDT
(In reply to Lars Vogel from comment #27)
> Jonah, creating / updating one configuration per library seems very complex.

Patches welcome :-) It is a very good idea. 

There are some additional requirements (CQ/clearlydefined tracking) that need to be resolved to make it happen (dash license tool uses the iplog info in Orbit to track this info). Fortunately for now the process while it creates many files, the new files are created with a single command line to EBR. 

This can't help remind me of one of my favourite comics: 
https://i0.wp.com/www.burkard.it/wp-content/uploads/2014/06/no-thanks-were-too-busy.jpg?w=600&ssl=1
Comment 29 Eclipse Genie CLA 2022-04-05 11:47:51 EDT
New Gerrit change created: https://git.eclipse.org/r/c/orbit/orbit-recipes/+/192517
Comment 31 Eclipse Genie CLA 2022-04-06 00:04:06 EDT
New Gerrit change created: https://git.eclipse.org/r/c/orbit/orbit-recipes/+/192525
Comment 32 Eclipse Genie CLA 2022-04-06 10:07:04 EDT
New Gerrit change created: https://git.eclipse.org/r/c/orbit/orbit-recipes/+/192539
Comment 35 Eclipse Genie CLA 2022-04-10 16:16:00 EDT
New Gerrit change created: https://git.eclipse.org/r/c/orbit/orbit-recipes/+/192613
Comment 37 Jonah Graham CLA 2022-04-27 09:24:49 EDT
Thanks to a lot of help with new recipes, the git based recipes are in better shape and contain the dependencies necessary for interested projects. There are still some CVS defined bundles being consumed by platform and other projects, as well as some git defined bundles that depend on CVS defined ones (at least at build time). These will be fixed as needed over time.

It would be nice to get https://git.eclipse.org/r/c/orbit/orbit-recipes/+/192539 merged, either by moving across any remaining bundles to Git, or by removing the bundles in Git that depend on CVS defined ones. As of now expending effort on these bundles seems somewhat useless as it is not clear that anyone is using those bundles.

e.g. apache xml-rpc was only used by Mylyn (in SimRel) which is not in simrel or active development anymore. So spending time get its dependencies all orbit friendly is not worth it, probably better to ditch it completely (which I am not doing!).