Full Text Bug Listing

Carlos Garcia Campos

Reported 2012-01-16 09:35:46 PST

If make is called without -j it works. I thibk the problem was introduced in r103920. Creating hashtable for ../Source/JavaScriptCore/runtime/StringPrototype.cpp CXX Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-AcceleratedCompositingContextClutter.lo CXX Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-AssertMatchingEnums.lo CXX Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-ChromeClientGtk.lo CXX Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-ContextMenuClientGtk.lo In file included from ../Source/WebCore/bindings/js/ScriptController.h:26:0, from ../Source/WebCore/page/Frame.h:40, from ../Source/WebCore/platform/gtk/PasteboardHelper.h:28, from ../Source/WebKit/gtk/WebCoreSupport/AssertMatchingEnums.cpp:29: ../Source/WebCore/bindings/js/JSDOMWindowShell.h:32:25: fatal error: JSDOMWindow.h: No such file or directory compilation terminated. In file included from ../Source/WebKit/gtk/webkit/webkitwebview.h:30:0, from ../Source/WebKit/gtk/WebCoreSupport/AcceleratedCompositingContext.h:26, from ../Source/WebKit/gtk/WebCoreSupport/AcceleratedCompositingContextClutter.cpp:21: ./DerivedSources/webkit/webkitdom.h:23:34: fatal error: webkit/WebKitDOMAttr.h: No such file or directory compilation terminated. In file included from ../Source/WebKit/gtk/webkit/webkitwebview.h:30:0, from ../Source/WebKit/gtk/WebCoreSupport/AcceleratedCompositingContext.h:26, from ../Source/WebKit/gtk/webkit/webkitwebviewprivate.h:26, from ../Source/WebKit/gtk/WebCoreSupport/ContextMenuClientGtk.cpp:29: ./DerivedSources/webkit/webkitdom.h:23:34: fatal error: webkit/WebKitDOMAttr.h: No such file or directory compilation terminated. In file included from ../Source/WebCore/bindings/js/ScriptController.h:26:0, from ../Source/WebCore/page/Frame.h:40, from ../Source/WebKit/gtk/webkit/webkitwebframeprivate.h:26, from ../Source/WebKit/gtk/WebCoreSupport/ChromeClientGtk.cpp:59: ../Source/WebCore/bindings/js/JSDOMWindowShell.h:32:25: fatal error: JSDOMWindow.h: No such file or directory compilation terminated. make[1]: *** [Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-AssertMatchingEnums.lo] Error 1 make[1]: *** Waiting for unfinished jobs.... make[1]: *** [Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-AcceleratedCompositingContextClutter.lo] Error 1 make[1]: *** [Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-ContextMenuClientGtk.lo] Error 1 make[1]: *** [Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-ChromeClientGtk.lo] Error 1 make[1]: Leaving directory `/home/cgarcia/src/git/gnome/WebKit/WebKitBuild/webkit-1.7.4/_build' make: *** [distcheck] Error 1

Martin Robinson

Comment 1 2012-02-06 11:48:30 PST

It seems this is likely caused by supplemental IDL changes. haraken, do you think you can take a look at this? http://trac.webkit.org/changeset/103920 http://trac.webkit.org/changeset/103729

Kentaro Hara

Comment 2 2012-02-06 16:31:50 PST

(In reply to comment #1) > It seems this is likely caused by supplemental IDL changes. haraken, do you think you can take a look at this? > > http://trac.webkit.org/changeset/103920 > http://trac.webkit.org/changeset/103729 I guess that the cause is that no dependency is written between $(webcore_built_sources) and $(webcore_sources). nodist_libWebCore_la_SOURCES = $(webcore_built_sources) libWebCore_la_SOURCES = $(webcore_sources) We need to guarantee that $(webcore_sources) are built _after_ $(webcore_built_sources). So I guess we can fix the issue by adding the following line: $(webcore_sources) : $(webcore_built_sources) I am making a patch. (But I guess that potentially this issue would have occurred before the [supplemental] change.)

Martin Robinson

Comment 3 2012-02-06 16:41:42 PST

(In reply to comment #2) > We need to guarantee that $(webcore_sources) are built _after_ $(webcore_built_sources). So I guess we can fix the issue by adding the following line: Typically the way to ensure this is to include auto-generated source files in BUILT_SOURCES. That's a hint to the build tools that the files should be generated during the first phase of the build.

Kentaro Hara

Comment 4 2012-02-06 16:48:29 PST

Created attachment 125730 [details] Patch

Kentaro Hara

Comment 5 2012-02-06 16:50:11 PST

Actually I could not reproduce the make -j error in my local GTK environment. If you think the patch is OK, I'd like to commit it. Otherwise, I am happy if you would check that the patch really fixes the make -j error in your GTK environment before committing it.

Martin Robinson

Comment 6 2012-02-06 16:53:30 PST

Comment on attachment 125730 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=125730&action=review This fix looks sane, but do you mind making the small change below before landing? > Source/WebCore/GNUmakefile.am:812 > + $(webkitgtk_gdom_built_sources) The GDOM sources are actually part of the WebKit build, not the WebCore one. It might make more sense to add it to BUILT_SOURCES in Source/WebCore/bindings/gobject/GNUmakefile.am.

Martin Robinson

Comment 7 2012-02-06 16:54:50 PST

Hrm. Actually it seems that webcore_built_sources is already added to BUILT_SOURCES in the toplevel GNUmakefile.am.

Kentaro Hara

Comment 8 2012-02-06 16:55:49 PST

(In reply to comment #7) > Hrm. Actually it seems that webcore_built_sources is already added to BUILT_SOURCES in the toplevel GNUmakefile.am. Then, the patch won't fix the issue...?

Martin Robinson

Comment 9 2012-02-06 17:42:04 PST

Carlos, this looks like the same issue that made me put $(BUILT_SOURCES) as a dependency for 'make docs' The issue, I think, is that you cannot do 'make distcheck' from a clean build. If you've already built (which works fine by the way), it should work.

Martin Robinson

Comment 10 2012-02-06 17:43:08 PST

I say this because it's failing for me before it event tries to build the tarball it creates. Here's the full build output: martin@thurgoodmcburdle:~/WebKit/WebKitBuild/Release$ ../../Tools/gtk/run-with-jhbuild make -j12 distcheck GEN DerivedSources/WebCore/InspectorBackendDispatcher.cpp { test ! -d "webkit-1.7.5" || { find "webkit-1.7.5" -type d ! -perm -200 -exec chmod u+w {} ';' && rm -fr "webkit-1.7.5"; }; } test -d "webkit-1.7.5" || mkdir "webkit-1.7.5" make \ top_distdir="webkit-1.7.5" distdir="webkit-1.7.5" \ dist-hook make[1]: Entering directory `/home/martin/WebKit/WebKitBuild/Release' GEN DerivedSources/WebCore/supplemental_dependency.tmp GEN DerivedSources/webkit/WebKitDOMObject.h GEN DerivedSources/webkit/WebKitDOMCustom.h GEN DerivedSources/webkit/WebKitDOMEventTarget.h GEN DerivedSources/webkit/webkitdomdefines.h GEN DerivedSources/webkit/webkitdom.h GEN stamp-webkitmarshal.cpp CXX Source/WebCore/html/libWebCore_la-ValidationMessage.lo CXX Source/WebCore/html/libWebCore_la-URLInputType.lo CXX Source/WebCore/html/libWebCore_la-ValidityState.lo CXX Source/WebCore/html/libWebCore_la-WeekInputType.lo ../../Source/WebCore/html/ValidationMessage.cpp:34:30: fatal error: CSSPropertyNames.h: No such file or directory compilation terminated. In file included from ../../Source/WebCore/dom/EventTarget.h:36:0, from ../../Source/WebCore/dom/Node.h:29, from ../../Source/WebCore/dom/ContainerNode.h:28, from ../../Source/WebCore/dom/Document.h:34, from ../../Source/WebCore/dom/Element.h:29, from ../../Source/WebCore/dom/StyledElement.h:28, from ../../Source/WebCore/html/HTMLElement.h:26, from ../../Source/WebCore/html/FormAssociatedElement.h:27, from ../../Source/WebCore/html/HTMLFormControlElement.h:27, from ../../Source/WebCore/html/HTMLFormControlElementWithState.h:27, from ../../Source/WebCore/html/HTMLTextFormControlElement.h:28, from ../../Source/WebCore/html/HTMLInputElement.h:27, from ../../Source/WebCore/html/URLInputType.cpp:34:

Kentaro Hara

Comment 11 2012-02-06 18:08:39 PST

(In reply to comment #9) > If you've already built (which works fine by the way), it should work. Yeah, maybe that is just because the generated files are already there. But why isn't the dependency from $(webcore_sources) to $(webcore_built_sources) guaranteed even if we add $(webcore_built_sources) to BUILT_SOURCES?

Martin Robinson

Comment 12 2012-02-06 18:14:32 PST

(In reply to comment #11) > Yeah, maybe that is just because the generated files are already there. But why isn't the dependency from $(webcore_sources) to $(webcore_built_sources) guaranteed even if we add $(webcore_built_sources) to BUILT_SOURCES? I'm still looking into this, but it may be because the $(BUILT_SOURCES) niceties only come when you are building the 'all' target.

Martin Robinson

Comment 13 2012-02-06 19:21:48 PST

(In reply to comment #12) > I'm still looking into this, but it may be because the $(BUILT_SOURCES) niceties only come when you are building the 'all' target. I confirmed locally that 'make -j12 distcheck' worked fine after running 'make -j12' first. Perhaps the problem Carlos is seeing is a different one, but I can't reproduce it right now.

Priit Laes (IRC: plaes)

Comment 14 2012-02-08 11:44:52 PST

(In reply to comment #13) > (In reply to comment #12) > > > I'm still looking into this, but it may be because the $(BUILT_SOURCES) niceties only come when you are building the 'all' target. > > I confirmed locally that 'make -j12 distcheck' worked fine after running 'make -j12' first. Perhaps the problem Carlos is seeing is a different one, but I can't reproduce it right now. One way to get build error is by adding --disable-dependency-tracking to configure (this used to work also before the supplemental IDL change stuff).

Carlos Garcia Campos

Comment 15 2012-02-12 04:38:40 PST

It still happens to me even with a normal make -j4 (no distcheck) in a clean WebKitBuild dir. CXX Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-ChromeClientGtk.lo In file included from ../Source/WebCore/bindings/js/ScriptController.h:26:0, from ../Source/WebCore/page/Frame.h:40, from ../Source/WebCore/platform/gtk/PasteboardHelper.h:28, from ../Source/WebKit/gtk/WebCoreSupport/AssertMatchingEnums.cpp:29: ../Source/WebCore/bindings/js/JSDOMWindowShell.h:32:25: error fatal: JSDOMWindow.h: No existe el archivo o el directorio compilación terminada. In file included from ../Source/WebKit/gtk/webkit/webkitwebview.h:30:0, from ../Source/WebKit/gtk/WebCoreSupport/AcceleratedCompositingContext.h:27, from ../Source/WebKit/gtk/WebCoreSupport/AcceleratedCompositingContextGL.cpp:20: ./DerivedSources/webkit/webkitdom.h:23:34: error fatal: webkit/WebKitDOMAttr.h: No existe el archivo o el directorio compilación terminada. In file included from ../Source/WebKit/gtk/webkit/webkitwebview.h:30:0, from ../Source/WebKit/gtk/WebCoreSupport/AcceleratedCompositingContext.h:27, from ../Source/WebKit/gtk/WebCoreSupport/AcceleratedCompositingContextClutter.cpp:21: ./DerivedSources/webkit/webkitdom.h:23:34: error fatal: webkit/WebKitDOMAttr.h: No existe el archivo o el directorio compilación terminada. In file included from ../Source/WebCore/bindings/js/ScriptController.h:26:0, from ../Source/WebCore/page/Frame.h:40, from ../Source/WebKit/gtk/webkit/webkitwebframeprivate.h:26, from ../Source/WebKit/gtk/WebCoreSupport/ChromeClientGtk.cpp:60: ../Source/WebCore/bindings/js/JSDOMWindowShell.h:32:25: error fatal: JSDOMWindow.h: No existe el archivo o el directorio So I guess it fails always running distcheck, not because distcheck makes anything different, but because it always builds on a clean build dir. Just running make -j4 again after the failure makes the build to continue, but that's not possible with distcheck, because every make distcheck starts from scratch

Kentaro Hara

Comment 16 2012-02-12 17:24:02 PST

(In reply to comment #15) > It still happens to me even with a normal make -j4 (no distcheck) in a clean WebKitBuild dir. > So I guess it fails always running distcheck, not because distcheck makes anything different, but because it always builds on a clean build dir. > Just running make -j4 again after the failure makes the build to continue, but that's not possible with distcheck, because every make distcheck starts from scratch I could not reproduce it maybe because my machine has only 2 CPUs. Anyway the cause is that it is not guaranteed that $(webcore_sources) are built _after_ $(webcore_built_sources). I am not sure how to write the dependency correctly in automake, but would you try to add the following line in WebCore/GNUMakefile.am and see what happens? $(webcore_sources) : $(webcore_built_sources)

Carlos Garcia Campos

Comment 17 2012-02-13 00:41:10 PST

(In reply to comment #16) > I could not reproduce it maybe because my machine has only 2 CPUs. Anyway the cause is that it is not guaranteed that $(webcore_sources) are built _after_ $(webcore_built_sources). I am not sure how to write the dependency correctly in automake, but would you try to add the following line in WebCore/GNUMakefile.am and see what happens? > > $(webcore_sources) : $(webcore_built_sources) It doesn't help :-( I'm not sure the problem is that webcore_sources depends on webcore_built_sources. In the Makefile the all: rule depends on BUILT_SOURCES, so they should all be generated before anything else. The problem is that libwebkitgtk is built before libWebCore, since it depends on it at link time, but libwebkitgtk depends on webcore generated sources (at least the headers). When the compilation starts, BUILT_SOURCES are indeed created, since it's a dependency of all: The first file generated is the supplemental file: GEN DerivedSources/WebCore/supplemental_dependency.tmp Then it generates the libwebkitgtk built sources, webkit2 built sources, JavaScriptCore built sources, and then it generates some of the webcore built sources: GEN DerivedSources/WebCore/CSSGrammar.cpp GEN DerivedSources/WebCore/CSSPropertyNames.h GEN DerivedSources/WebCore/CSSValueKeywords.h GEN DerivedSources/WebCore/ColorData.cpp GEN DerivedSources/WebCore/EventFactory.cpp GEN DerivedSources/WebCore/EventHeaders.h GEN DerivedSources/WebCore/EventInterfaces.h GEN DerivedSources/WebCore/EventTargetHeaders.h GEN DerivedSources/WebCore/EventTargetInterfaces.h GEN DerivedSources/WebCore/ExceptionCodeDescription.cpp GEN DerivedSources/WebCore/ExceptionInterfaces.h GEN DerivedSources/WebCore/ExceptionHeaders.h GEN DerivedSources/WebCore/HTMLNames.h GEN DerivedSources/WebCore/HTMLEntityTable.cpp GEN DerivedSources/WebCore/InjectedScriptSource.h GEN DerivedSources/WebCore/InspectorBackendDispatcher.cpp GEN DerivedSources/WebCore/InspectorProtocolVersion.h GEN DerivedSources/WebCore/SVGElementFactory.cpp GEN DerivedSources/WebCore/MathMLElementFactory.cpp GEN DerivedSources/WebCore/MathMLNames.cpp GEN DerivedSources/WebCore/UserAgentStyleSheets.h GEN DerivedSources/WebCore/WebKitFontFamilyNames.h GEN DerivedSources/WebCore/XLinkNames.cpp GEN DerivedSources/WebCore/XMLNames.cpp GEN DerivedSources/WebCore/XMLNames.h GEN DerivedSources/WebCore/XMLNSNames.cpp GEN DerivedSources/WebCore/XMLNSNames.h GEN DerivedSources/WebCore/XMLViewerCSS.h GEN DerivedSources/WebCore/XMLViewerJS.h GEN DerivedSources/WebCore/XPathGrammar.cpp GEN DerivedSources/ANGLE/glslang.cpp GEN DerivedSources/ANGLE/glslang_tab.cpp These are the ones that have an explicit rule in the Makefile and don't depend on the supplemental temp file. The rule for webcore built sources starting with JS is not executed, I guess it's because all dependencies already exist. I have tried several changes in the makefile and I haven't managed to make it build.

Carlos Garcia Campos

Comment 18 2012-02-13 00:44:26 PST

When running make again after it fails, all DerivedSources/webkit/WebKitDOM* files are generated and then all DerivedSources/WebCore/JS* and the build completes successfully.

Kentaro Hara

Comment 19 2012-02-14 17:10:28 PST

(In reply to comment #17) > These are the ones that have an explicit rule in the Makefile and don't depend on the supplemental temp file. The rule for webcore built sources starting with JS is not executed, I guess it's because all dependencies already exist. Carlos: Sorry for the trouble. In my understanding, the supplemental change should not have changed dependencies in makefile (but actually it has changed the dependencies). Just a random thought, but maybe is it related to .SECONDARY? What happens if we remove two ".SECONDARY"s from $(SUPPLEMENTAL_DEPENDENCY_FILE) and DerivedSources/WebCore/JS%.h?

Carlos Garcia Campos

Comment 20 2012-02-15 00:43:21 PST

(In reply to comment #19) > (In reply to comment #17) > > These are the ones that have an explicit rule in the Makefile and don't depend on the supplemental temp file. The rule for webcore built sources starting with JS is not executed, I guess it's because all dependencies already exist. > > Carlos: Sorry for the trouble. In my understanding, the supplemental change should not have changed dependencies in makefile (but actually it has changed the dependencies). > > Just a random thought, but maybe is it related to .SECONDARY? What happens if we remove two ".SECONDARY"s from $(SUPPLEMENTAL_DEPENDENCY_FILE) and DerivedSources/WebCore/JS%.h? I think I already tried to remove the .SECONDARY rules and it didn't help. Btw, .SECONDARY rule with no deps marks all targets in the makefile as secondary, so we don't need two .SECONDARY rules there

Martin Robinson

Comment 21 2012-02-21 23:19:09 PST

I just spent hours debugging this issue, but I think I've managed to produce a fix. As far as I can tell the problem is this: 1. BUILT_SOURCES doesn't do anything except put certain targets at the front of the source list by making all depend on $(BUILT_SOURCES). 2. If those targets cannot be built, because a dependency hasn't been generated yet, GNUmake is smart enough to build them later. 3. Once this happens GNUmake will continue walking down the target list past BUILT_SOURCES into non-autogenerated sources. Ideally $(BUILT_SOURCES) should be built serially before the rest of the 'all' target dependencies, but autotools doesn't do that. It may not even be possible without a new hook in GNUmake. This is even more of an issue because generating the supplemental IDL information is really slow. It's often the case that GNUmake will try to build a file that depends on a generated-by-IDL file before the supplemental IDL target finishes. The solution here, as with most problems in GNUmake is to properly specify dependencies. Patch incoming.

Kentaro Hara

Comment 22 2012-02-21 23:28:07 PST

(In reply to comment #21) > I just spent hours debugging this issue, but I think I've managed to produce a fix. Ah, really great, thanks! I also just spent hours debugging this issue today but have not yet reached the solution:-(

Martin Robinson

Comment 23 2012-02-22 10:56:20 PST

Committed r108523: <http://trac.webkit.org/changeset/108523>

Kentaro Hara

Comment 24 2012-02-22 15:44:45 PST

Really thanks!

Philippe Normand

Comment 25 2012-02-28 03:07:37 PST

If I disable WebKit2 build, jsc-3 is created but the build fails later on: CXXLD Programs/jsc-3 CXXLD libjavascriptcoregtk-3.0.la CXXLD Libraries/libTestRunnerInjectedBundle.la libtool: link: cannot find the library `libjavascriptcoregtk-3.0.la' or unhandled argument `libjavascriptcoregtk-3.0.la' make[1]: *** [Programs/minidom] Error 1 make[1]: *** Waiting for unfinished jobs.... libtool: link: cannot find the library `libjavascriptcoregtk-3.0.la' or unhandled argument `libjavascriptcoregtk-3.0.la' make[1]: *** [Programs/jsc-3] Error 1 make[1]: Leaving directory `/home/phil/gst/jhbuild/build/WebKit/WebKitBuild/tmp/Release' make: *** [all] Error 2 Could this whole issue be related with r108523 ?

Philippe Normand

Comment 26 2012-02-28 03:07:53 PST

wrong bug, sorry

Attachments
Patch (1.54 KB, patch) 2012-02-06 16:48 PST, Kentaro Hara	no flags	Details Formatted Diff Diff
View All Add attachment proposed patch, testcase, etc.

Bug 76388