Community
Participate
Working Groups
Currently on marketplace you can see that solution provider are creating seperate solutions for different Eclipse releases. We want to fix that by adding a functionality to MPC to filter out incompatible software. With this bug, I would like to discuss how we should be managing compatibility. Before I can close this bug I need an answer for the following questions: 1- Granularity of release: Should we limit compatibility to a major release or should we also include milestones and service releases? 2- What should be the default compatibility setting for a solution that was submitted before we start requiring our users to include compatibility information when submitting a new solution? 3- Should we assume that a listing will be compatible with a new release or should the solution provider certify that his listing will work with the new release? My biggest fear/concern with this new feature is that that number of listings served by marketplace could shrink after each major release of Eclipse unless we assume that everyone is going to test and update their listing information before each major release.
> 1- Granularity of release: Should we limit compatibility to a major release or > should we also include milestones and service releases? I think we should keep this as simple as possible and limit it to major releases. There might be the odd outlier somewhere that is compatible with a release but not the following SRs, but this should be very rare and I wouldn't burden everyone else with a more complicated workflow just because of it. The other way around shouldn't be a problem either: if something isn't compatible with the release, it is just flagged as incompatible. If it becomes compatible once a SR has come out, they can flag it as compatible with the release. This works due to the simrel repositories containing both the release and the updates for SR1 at this point. So even if the entry needs a SR1 version and the client is still on the original release version, they'll just get the necessary update alongside the install. I think it would also be okay to restrict this to one continuous version range, which probably makes the implementation - and especially the ui - a bit simpler. I don't imagine that there are any plug-ins that are compatible with e.g. Indigo and Kepler, but not Juno. I would present the release train name in the admin ui and map that internally to the expected version ranges for the versions transmitted by MPC. > 2- What should be the default compatibility setting for a solution that was > submitted before we start requiring our users to include compatibility > information when submitting a new solution? I would suggest proceeding in stages here: i. consider all legacy entries as compatible with all releases ii. once the feature is ready, require owners to specify compatibility for new entries or for updates to existing ones iii. some time after Mars, notify owners of legacy entries without version info to update them (rationale: hopefully most of the active entries will have been updated by then anyway) (iv. some time after that, flag legacy entries as incompatible) > 3- Should we assume that a listing will be compatible with a new release or > should the solution provider certify that his listing will work with the new > release? I think we should consider entries as incompatible with new major releases by default. Otherwise nobody will spend any time on specifying that information and the data will become useless pretty quickly. That's also a reason - in addition to usability - for not letting the solution provider specify an OSGi version range in textual form. I don't think it would be a good idea to let people specify open ranges or arbitrary upper version bounds. But we should probably poke the providers with a mail reminding them to update the info on each release. > My biggest fear/concern with this new feature is that that number of listings > served by marketplace could shrink after each major release of Eclipse unless we > assume that everyone is going to test and update their listing information > before each major release. I think the number will shrink naturally for a short time after each major release and then go up again quickly as more providers announce compatibility - except for "dead" entries, which I wouldn't consider a bad thing. I think that's overall preferable to having a larger number of entries, of which a good percentage isn't installable anyway, causing user frustration. And if the filter can be disabled in the client (bug 385883, 3.3), people can still install entries, but they will be aware that they're off into uncharted territory. Let me suggest one more point for the list: 4- It shouldn't be difficult to write a headless server-side eclipse job (running e.g. on our hudson or the marketplace server itself) that can check which entries are compatible to which releases based on their actual dependencies and installability. I can look into that if that's interesting. This would be a somewhat slow and resource-hungry process though, doing all the work that normally happens in Eclipse between selecting an entry for installation and clicking "finish" (i.e. it would do the whole resolution and planning, but not actually download any plug-in files). We could use that either in bulk to check compatibility e.g. for legacy entries or for new releases (that would run for quite some time though...). It could also be used to check new/changed entries, or to periodically pick a random entry to check.
*** Bug 416532 has been marked as a duplicate of this bug. ***
How this relates to listing names should be considered. One common convention where a different listings are required is "Product for Release". This serves two purposes: 1. Differentiate listings in MPC. 2. Differentiate listings on the website, such as for listing management. If after this change, Marketplace is filtering the catalog returned to MPC by Eclipse version, it would be nice to be able to drop the "for Release" portion of the listing names. However, this would cause a problem for #2. Perhaps the website should show the compatible releases in the listing summary block that you get when browsing/searching?
(In reply to Konstantin Komissarchik from comment #3) > How this relates to listing names should be considered. One common > convention where a different listings are required is "Product for Release". I was hoping this change would negate the need to have this type of naming. My hope is you just need one listing for each product. > This serves two purposes: > > 1. Differentiate listings in MPC. > 2. Differentiate listings on the website, such as for listing management. If MPC is smart to only show the listing that is compatible, why do you need to differentiate? > > If after this change, Marketplace is filtering the catalog returned to MPC > by Eclipse version, it would be nice to be able to drop the "for Release" > portion of the listing names. However, this would cause a problem for #2. Can you explain the value of the "for Release" naming?
> I was hoping this change would negate the need to have this type of naming. > My hope is you just need one listing for each product. I might be misunderstanding the planned change... Is the plan to allow version tagging at solution level or is the plan to allow multiple version-tagged variants of feature sets, descriptions, screen shots, etc. within one solution.
I disagree with using this to force solution providers to explicitly state compatibility with every major release. Open ranges should be supported, including the ultimate open range of everything. There are many solutions that use only public API and remain compatible indefinitely, so we'd be just adding unnecessary busy work for them. Keep in mind that some of the smaller ones are only sporadically maintained when a bug report comes in. That doesn't make them dead solutions. Verifying validity of listings in automated fashion, for multiple versions of Eclipse, would be a very valuable service on its own accord that would catch version compatibility and other problems. If this was run regularly and failures sent to solution providers, they can adjust the listings accordingly at that point.
(In reply to Konstantin Komissarchik from comment #6) > I disagree with using this to force solution providers to explicitly state > compatibility with every major release. Open ranges should be supported, > including the ultimate open range of everything. I agree. :-) When we migrate to the new system the default will be that the listing is compatible with everything. When a new release comes out the default will be the listing is compatible. However, in 2+ releases if the listing has not been updated the default will be not compatible. For active listings, this should not be a problem. For listing that aren't actively maintained, well they will need to become more active; at least once a year. > > Verifying validity of listings in automated fashion, for multiple versions > of Eclipse, would be a very valuable service on its own accord that would > catch version compatibility and other problems. If this was run regularly > and failures sent to solution providers, they can adjust the listings > accordingly at that point. We have the error reports already. Not sure how we can automatically detect if an error is due to incompatibility?
(In reply to Konstantin Komissarchik from comment #5) > > I was hoping this change would negate the need to have this type of naming. > > My hope is you just need one listing for each product. > > I might be misunderstanding the planned change... Is the plan to allow > version tagging at solution level or is the plan to allow multiple > version-tagged variants of feature sets, descriptions, screen shots, etc. > within one solution. The versioning is at the feature ID and update site level. It only deals with what bits to install.
> For listing that aren't actively maintained, well they will need to become more > active; at least once a year. Why is this an important goal? I concur with those that say that this will result in a drastic reduction in solutions available on the Marketplace for many months after a new release is out. Many of the smaller solution will likely disappear completely even if there is nothing wrong with them. A much better approach is an automated test system that hides the listing if the test has failed. This would catch version incompatibilities and other issues. > We have the error reports already. Error reports from end users are not terribly useful as those installations aren't performed in a controlled environment and thus the problems are often not with the listing. The two most common causes that I've seen is a network glitch and installing an incompatible mix of solutions.
> The versioning is at the feature ID and update site level. In that case, disregard my comments on naming. They only apply to the other alternative. Is there a mock up of the admin UI available at this point? > It only deals with what bits to install. What about description and screen shots? If the description touches on the available features, it would have to include version-specific wording to compensate, such as "available in Luna and later". A similar thing cannot be done for screen shots. Since the UI can change over time, to avoid confusing users, many solutions would have to drop the screen shots completely. Thought should also be given to version-tagging comments as a comment regarding some new feature can be pretty confusing for a user of an older version of Eclipse where the feature is not available.
(In reply to Konstantin Komissarchik from comment #9) > Why is this an important goal? I concur with those that say that this will > result in a drastic reduction in solutions available on the Marketplace for > many months after a new release is out. Many of the smaller solution will > likely disappear completely even if there is nothing wrong with them. Since entries would be compatible with a new release by default, I don't see why many would disappear after a new release. Also, before we would turn off old entries that haven't been updated to declare their compatibility explicitly, there would be notification to the entry's owner well in advance. > A much better approach is an automated test system that hides the listing if > the test has failed. This would catch version incompatibilities and other > issues. We have discussed this and it might be something to investigate in the future (maybe even before any entries are switched off through other means). But we wanted to keep it simple for now. > > > We have the error reports already. > > Error reports from end users are not terribly useful as those installations > aren't performed in a controlled environment and thus the problems are often > not with the listing. The two most common causes that I've seen is a network > glitch and installing an incompatible mix of solutions. We also have reports of install successes. So if an entry has very few install successes and very many errors for a given release, it could be a strong candidate for an incompatibility. The message of the install error (usually a version constraint conflict for some bundles) could be an indication, too.
> Since entries would be compatible with a new release by default, I don't see > why many would disappear after a new release. I was responding to the statement made that eventually all solutions have to declare compatibility or not be listed for a given release. My point is that there is no value to be gained from forcing this issue. Open version ranges should be supported indefinitely. Let this feature be something solution providers can take advantage of as needed instead of burdening everyone with extra process.
(In reply to Konstantin Komissarchik from comment #12) > > Since entries would be compatible with a new release by default, I don't see > > why many would disappear after a new release. > > I was responding to the statement made that eventually all solutions have to > declare compatibility or not be listed for a given release. The idea is that solutions will always be considered compatible for a new release if they were compatible for the last one. But as soon as you update your entry, you'll be forced to confirm that assumption (or adjust accordingly). If you don't do that for a very long time (those 2+ years), we might consider changing that assumption for this particular entry. But maybe there'll be a smarter automatism by then. > My point is that there is no value to be gained from forcing this issue. > Open version ranges should be supported indefinitely. Let this feature be > something solution providers can take advantage of as needed instead of > burdening everyone with extra process. The value is that false positives, i.e. incompatible entries showing up as installable for the user, hurt the user experience.
(In reply to Konstantin Komissarchik from comment #10) > > It only deals with what bits to install. > > What about description and screen shots? If the description touches on the > available features, it would have to include version-specific wording to > compensate, such as "available in Luna and later". A similar thing cannot be > done for screen shots. Since the UI can change over time, to avoid confusing > users, many solutions would have to drop the screen shots completely. Sorry, I had a longer reply to this, but lost it in a bugzilla snafu. In essence, I don't think we should overcomplicate the entry. And I don't see anybody wanting to maintain n+1 variations of their entry. Other app stores certainly do fine with the odd "only available in..." in the description. And I think that's easy enough for the entry maintainer and at their full discretion. The added value of detailed texts/screenshots per version is very low IMHO and mostly owed to our engineers' precision mindset (which I totally get, my initial reaction, too, was that we might need specific details - but thinking about it a bit more, I think they do more harm than good). > Thought should also be given to version-tagging comments as a comment > regarding some new feature can be pretty confusing for a user of an older > version of Eclipse where the feature is not available. Same applies here. Also, since we're only discerning the install bits, there's nothing in the web UI to go by wrt the selected version. And in MPC, where we do have that information, we don't have support for comments at the moment.
> The idea is that solutions will always be considered compatible for a new > release if they were compatible for the last one. Could you go into more detail how you expect version compatibility to be specified? If my entry is marked specifically for Luna, I would hope that it doesn't show up in Mars. That is, it should be possible to specify Luna or Luna+ depending on solution needs. Using solutions that I maintain as an example... For Sapphire, I have v8.x that supports Indigo+ and v9 that supports Luna+, so I should be able to tag in Marketplace that v8.x is for Indigo-Kepler and v9 is Luna+. For Oracle tools, on the other hand, it needs to one release only. The Luna tools should not show up for Mars, for instance. The version specification should support single versions, closed ranges or lists of versions, and open ranges. > The value is that false positives, i.e. incompatible entries showing up > as installable for the user, hurt the user experience. What hurts the user experience is non-working entries. The reason why they aren't working isn't relevant. The proposed approach will disable a small fraction of those that should be disabled, but also ensnare many functional entries and cause more work for every solution provider. It would do more harm that good.
On the topic of removing entries on MPC automatically based on missing metadata then I think the better approach is: A) provide options for specifying compatability with release trains/eclipse versions. B) have automatic monitoring of when an MPC entry creates more errors than installs then contact the MPC entry owner. C) If owner in #B does not respond withing 14 days, forcefully apply the the metadata available in A to a release train that is known to work D) If C does not help, remove/disable the entry. All of this (except C which should happen rarely over time) can be automated allows for those useful plugins that continue to be compatible and exist to not be removed without valid reason.
This task has been completed and we are now filtering out incompatible marketplace listings.
