Bug 170500

Ok, this is my initial proposal: pixman: 0.32.6 -> 0.34.0 cairo: 1.14.2 -> 1.14.8 freetype: 2.4.11 -> 2.6.3 harfbuzz: 1.3.3 -> 1.4.2 libffi: 3.1 (do we really need this in the internal jhbuild? glib depends on 3.0 I think we can remove this one) gdk-pixbuf: 2.30 -> 2.36.6 librsvg: 2.36.1 -> 2.40.16 gtk+: 3.16.4 -> 3.22.11 glib: 2.44.1 -> 2.52.0 glib-networking: 2.41.4 -> 2.50.0 libsoup: 2.57.1 (this is the latest already) fontconfig: 2.11.1 (this is recent enough) gnome-icon-theme: 3.2.1 (do we really need this) gnome-icon-theme-symbolic: 3.2.1 (do we really need this) gnome-themes-standard: 3.6.0 (do we really need this) atk: 2.15.4 -> 2.24.0 at-spi2-core: 2.15.4 -> 2.24.0 at-spi2-atk: 2.15.4 -> 2.24.0 libxml2: 2.9.1 (do we really need this) libxslt: 1.1.29 (do we really need this) orc: 0.4.17 gstreamer: 1.8.0 -> 1.10.4 gst-plugins-base: 1.8.0 -> 1.10.4 gst-plugins-good: 1.8.0 -> 1.10.4 gst-plugins-bad: 1.8.0 -> 1.10.4 gst-libav: 1.8.0 -> 1.10.4 xserver: 1.16.4 -> 1.19.2 wayland: 1.8.1 -> 1.12.0 weston: 1.8.0 -> 1.12.0 gtk-doc: 1.25 (this is recent enough) libdrm: 2.4.65 -> 2.4.74 mesa: 11.0.6 -> 13.0.6 llvm: 3.7.0 -> 3.8.0 gsettings-desktop-schemas: 3.16 (do we really need this) shared-mime-info: 1.5 -> 1.8 icu: 55.1 -> 57.1 pango: 1.36.8 -> 1.40.4 libinput: 1.2.4 -> 1.6.3 I wanted to bump all at once to check the tests, but maybe we can do it in two steps and leave gst related modules to a second patch.

The plan looks good to me. I have added a few comments below. (In reply to Carlos Garcia Campos from comment #1) > Ok, this is my initial proposal: > > [...] > libffi: 3.1 (do we really need this in the internal jhbuild? glib depends on > 3.0 I think we can remove this one) This can go away. Even Debian Stable has version 3.1 of libffi nowadays: https://packages.debian.org/jessie/libffi-dev > [...] > librsvg: 2.36.1 -> 2.40.16 Just in case someone else wonders why not using 2.41.0: that version is the first one including some parts rewritten in Rust. Staying in 2.40.x saves us from building a Rust toolchain and from requiring it installed system-wide. IMHO for now 2.40.16 is a good tradeoff between updating and not requiring one extra complex dependency. > [...] > gnome-icon-theme: 3.2.1 (do we really need this) > gnome-icon-theme-symbolic: 3.2.1 (do we really need this) > gnome-themes-standard: 3.6.0 (do we really need this) Using GTK+ from JHBuild does not fall-back well to using the versions of these packages installed system-wide. I think that's the reason why we have been keeping those in the moduleset. > [...] > libxml2: 2.9.1 (do we really need this) Debian stable has this version already (exactly 2.9.1, actually). I would remove it from the JHBuild moduleset. > libxslt: 1.1.29 (do we really need this) Our CMake files check for version 1.1.7, and Debian stable has 1.1.28. Most likely we can remove it from the JHBuild. > [...] > orc: 0.4.17 Is there any reason to not update this? Could it be removed, even? (Debian stable has 0.4.22, I guess we have to keep it in the JHBuild if changing it could affect the output from tests, but that seems unlikely. Dunno, I am not sure about this one.)

I'll split it in more parts, there are too many failures to track at once, I'm getting ~5600 failures :-P

(In reply to Carlos Garcia Campos from comment #3) > I'll split it in more parts, there are too many failures to track at once, > I'm getting ~5600 failures :-P Maybe most of them are only because of freetype, though. I'll check it.

Expected to fail, but passed: (127) Regressions: Unexpected text-only failures (5443) Regressions: Unexpected image-only failures (24)

Feel free to upload the WIP patch here. I can help to test it.

Created attachment 306375 [details] WIP This is the complete patch

I get this(In reply to Carlos Garcia Campos from comment #7) > Created attachment 306375 [details] > WIP > > This is the complete patch I also get around ~5k new failures: Regressions: Unexpected text-only failures (5507) A quick look at the results shows that there are many tests that just need a rebaseline (just some pixel differences on the RenderBlocks). But many others are clear real failures. And having to manually go through 5k results to check which ones need a rebaseline or not is crazy. I'm thinking in creating some utility that can automatically tell is the test just need a rebaseline (by checking that the diff only changes pixel numbers on the render blocks) or if it needs manual inspection because it may be a potential failure. I wonder if we already have something like that??? How the Mac port handles rebaselines when they release a new version of OSX?? Any idea?

What we need to do is upgrade the libraries one-by-one so we can figure out if a particular library update broke tests and get the updates that don't break tests out of the way. I bet only three or four of those libraries actually break any tests. Trying to take them all on all at the same time is clearly not going to work. Maybe it would work in the future if we upgrade more regularly, but doing several years' worth of upgrades for all dependencies all at the same time is just not going to work.

(In reply to Michael Catanzaro from comment #9) > What we need to do is upgrade the libraries one-by-one so we can figure out > if a particular library update broke tests and get the updates that don't > break tests out of the way. Yes, that's the plan, and the reason why this is a meta bug and the patch just a reference to be split. > I bet only three or four of those libraries > actually break any tests. cairo and freetype are my bets. Other deps like libxml, gtk+, etc. will introduce failures for sure, not not that many. > Trying to take them all on all at the same time is > clearly not going to work. We already realized, see comment 3. > Maybe it would work in the future if we upgrade > more regularly, but doing several years' worth of upgrades for all > dependencies all at the same time is just not going to work. Indeed, we should upgrade at least once per cycle, at the very beginning.

(In reply to Michael Catanzaro from comment #9) > What we need to do is upgrade the libraries one-by-one so we can figure out > if a particular library update broke tests and get the updates that don't > break tests out of the way. I bet only three or four of those libraries > actually break any tests. Trying to take them all on all at the same time is > clearly not going to work. Maybe it would work in the future if we upgrade > more regularly, but doing several years' worth of upgrades for all > dependencies all at the same time is just not going to work. Broking them one by one is not going to cut the 5000 layout failures down. Only split them. So I still see the need for some tool that can help to automatically difference the diffs that are just minor pixel differences on the render block fom something else. This will prove useful as time passes because we intend to keep upgrading the moduleset much more frequently

What about this? https://trac.webkit.org/wiki/RebaselineServer

I've upgraded cairo and pixman and there aren't differences in test results. So, it was freetype alone the one causing more than 5000 failures. libxml2 is still pending too, there were regressions with 1.9.4 that has been recently fixed in git, so we could either switch to use git or wait for a new libxml2 release.

We can close this now.