Bug 42756

Summary:

JIT requires VM overcommit (particularly on x86-64), Linux does not by default support this without swap?

Product:

WebKit

Reporter:

donaldcallen

Component:

JavaScriptCore

Assignee:

Xan Lopez <xan.lopez>

Status:

RESOLVED FIXED

Severity:

Normal

CC:

ademar, ap, barraclough, camaradetux, darxus, edwin+webkit, ggaren, jens.timmerman, kevin-webkit, mrobinson, ojab, oliver, scottt.tw, uzytkownik2, xan.lopez, zherczeg

Priority:

Version:

528+ (Nightly build)

Hardware:

OS:

Linux

Attachments:

Description	Flags
gdb backtrace	none
Valgrind output	none
Segfault when running jsc without any argument (debug build)	none
Segfault when running jsc without any argument (debug build)	none
overcommit.diff	none
overcommit.diff	none
Patch	none

donaldcallen

Reported 2010-07-21 08:14:36 PDT

Created attachment 62181 [details] gdb backtrace I have been experiencing segfaults in memcpy using recent nightly builds. This problem occurs on only one of my 4 computers running Slackware 13.1. The machines are a Lenovo S10 workstation (4-core Q6600 processor, 4 Gb), a mini itx machine with Intel Atom D510 motherboard and 2 Gb, a Thinkpad X61 (Core 2 Duo, 2 Gb) and a Toshiba netbook (single core Atom N450, 2 Gb). The segfault I am reporting occurs only on the mini itx machine. I download the nightly build, plus the webkit slackbuild from slackbuild.org, edit the webkit.Slackbuild script to reflect the proper version number of the nightly build, and build the Slackbuild on the fast Lenovo workstation. If I install the resulting package on the workstation (or the netbook or the Thinkpad) and build uzbl or surf against it, either one will run. Do the same thing on the mini itx machine, and either one will segfault on startup. I tried building webkit on the mini itx machine (which takes a couple of hours on a machine that slow), installed it and built uzbl against that, and it segfaulted. I then copied *that* slackbuild to the workstation, installed it, built and installed uzbl against that version, and it worked on the workstation. The same is true of the version built on the workstation (interestingly, the two slackware packages are not identical, despite the two machines having identical package-sets) -- uzbl built against that works everywhere but the mini itx machine. If you are suspecting the hardware, all I can tell you is that everything else works correctly on the mini itx machine, including google chrome. There are absolutely no signs of sick hardware, and that machine is my primary machine -- I use it every day. I built a version of webkit (on the workstation) with CFLAGS changed from -O2 to -g and the slackbuild script modified not to strip the symbols from the executable. I then modified the uzbl-browser script to run uzbl-core under gdb. I am attaching a backtrace of the crash. I also modified the uzbl-browser script to run uzbl-core under valgrind. The output of that session is also attached.

Attachments
gdb backtrace (5.31 KB, text/plain) 2010-07-21 08:14 PDT, donaldcallen	no flags	Details
Valgrind output (3.74 KB, text/plain) 2010-07-21 08:17 PDT, donaldcallen	no flags	Details
Segfault when running jsc without any argument (debug build) (1.99 KB, text/plain) 2010-08-20 07:46 PDT, Adrien Nader	no flags	Details
Segfault when running jsc without any argument (debug build) (1.28 KB, text/plain) 2010-11-14 05:05 PST, ojab	no flags	Details
overcommit.diff (6.36 KB, patch) 2011-01-12 14:26 PST, Xan Lopez	no flags	Details Formatted Diff Diff
overcommit.diff (6.50 KB, patch) 2011-01-12 17:04 PST, Xan Lopez	no flags	Details Formatted Diff Diff
Patch (2.00 KB, patch) 2011-05-20 08:12 PDT, Xan Lopez	no flags	Details Formatted Diff Diff
Show Obsolete (2) View All Add attachment proposed patch, testcase, etc.

donaldcallen

Comment 1 2010-07-21 08:17:14 PDT

Created attachment 62182 [details] Valgrind output

Alexey Proskuryakov

Comment 2 2010-07-21 16:50:47 PDT

#0 0x00007ffff37f2e51 in memcpy () from /lib64/libc.so.6 #1 0x00007ffff7649dba in JSC::JIT::privateCompileCTIMachineTrampolines(WTF::RefPtr<JSC::ExecutablePool>*, JSC::JSGlobalData*, JSC::TrampolineStructure*) () from /usr/lib64/libwebkitgtk-1.0.so.0 #2 0x00007ffff7666400 in JSC::JIT::compileCTIMachineTrampolines(JSC::JSGlobalData*, WTF::RefPtr<JSC::ExecutablePool>*, JSC::TrampolineStructure*) () from /usr/lib64/libwebkitgtk-1.0.so.0 #3 0x00007ffff76ccdb3 in JSC::JSGlobalData::JSGlobalData(JSC::JSGlobalData::GlobalDataType, JSC::ThreadStackType) () from /usr/lib64/libwebkitgtk-1.0.so.0 #4 0x00007ffff76ccf13 in JSC::JSGlobalData::create(JSC::ThreadStackType) () from /usr/lib64/libwebkitgtk-1.0.so.0 #5 0x00007ffff76cd832 in JSC::JSGlobalData::createLeaked(JSC::ThreadStackType) () from /usr/lib64/libwebkitgtk-1.0.so.0 #6 0x00007ffff6b8e5f2 in WebCore::JSDOMWindowBase::commonJSGlobalData() () from /usr/lib64/libwebkitgtk-1.0.so.0

Adrien Nader

Comment 3 2010-08-20 03:21:42 PDT

I am experiencing the same problem on my computer with webkit-gtk 1.3.3. GtkLauncher fails and simply creating a web_view and using load_uri (no matter if the target exists or not, local or remote) makes it crash. g_thread_init(NULL); g_type_init(); gtk_init(&argc, &argv); WebKitWebView* wv = WEBKIT_WEB_VIEW(webkit_web_view_new ()); webkit_web_view_load_uri(wv, "http://google.com"); If I don't load any page, it doesn't crash. I'm also using slackware 13.1, custom-compiled kernel (2.6.35+, meaning git). gcc 4.4.4, glib2 2.22.5, glibc 2.11.1, gtk+ 2.18.9, libsoup 2.30.2, icu4c 4.4.1, webkit-gtk 1.3.3. Webkit was compiled without any non-default option (cflags=cxxflags="-O2 -fPIC"). I'm compiling on a quad-core (phenom ii x4 955) which has the same configuration (except the kernel) and running an a laptop (core2duo t5450). The quad-core is currently headless. I'm going to try several builds with different options (first, --disable-jit) and building on my laptop (it's only 5 times slower...) and will report back.

Adrien Nader

Comment 4 2010-08-20 05:49:50 PDT

Here are the results after a few builds. First, building on the laptop machine (the one running it) did *NOT* solve the problem. Then, of course, --disable-jit solved it. Finally, --disable-optimizations didn't solve it. Also, the 'jsc-1' binary, ran without any argument, crashes immediately. As I said, the build machine is headless so I can't really test webkit-gtk on it. However, trying jsc-1 on the build machine works without problem. (actually, I tried to create a web_view and load_uri a page, without creating a windows nor displaying anything and I got the expected "cannot open display" warning but no segfault) The backtrace for jsc-1 on the laptop is: #0 0x00007ffff50a4e51 in memcpy () from /lib64/libc.so.6 #1 0x00000000004d988f in JSC::JIT::privateCompileCTIMachineTrampolines(WTF::RefPtr<JSC::ExecutablePool>*, JSC::JSGlobalData*, JSC::TrampolineStructure*) () #2 0x00000000004f6570 in JSC::JIT::compileCTIMachineTrampolines(JSC::JSGlobalData*, WTF::RefPtr<JSC::ExecutablePool>*, JSC::TrampolineStructure*) () #3 0x000000000043f29c in JSC::JSGlobalData::JSGlobalData(JSC::JSGlobalData::GlobalDataType, JSC::ThreadStackType) () #4 0x000000000043f663 in JSC::JSGlobalData::create(JSC::ThreadStackType) () #5 0x0000000000408a77 in main () I'm also going to see if it can be reproduced in qemu, and if not, I can probably give a shell access if someone wants to try to debug and can't reproduce (unfortunately, not before a few days).

Zoltan Herczeg

Comment 5 2010-08-20 06:55:38 PDT

Is it happens in debug as well? Would be a good thing to see some line numbers. I expect missing frames on the stack since many funcions are inlined. (privateCompileCTIMachineTrampolines is the first function which generate machine code, perhaps it cannot allocate executable space) Address 0x27c is not stack'd, malloc'd or (recently) free'd This address is way too low...

Adrien Nader

Comment 6 2010-08-20 07:46:30 PDT

Created attachment 64955 [details] Segfault when running jsc without any argument (debug build)

Zoltan Herczeg

Comment 7 2010-08-20 09:07:11 PDT

(In reply to comment #6) > Created an attachment (id=64955) [details] > Segfault when running jsc without any argument (debug build) Hehe, as I suspected. Something with the allocator. Could you check this call successful on your machine (some printf is enough, mem.base() and mem.size() is important): 311 inline PassRefPtr<ExecutablePool> ExecutablePool::create(size_t n) 312 { 313 Allocation mem = systemAlloc(roundUpAllocationSize(n, JIT_ALLOCATOR_PAGE_SIZE)); 314 if (!mem) 315 return 0; 316 return adoptRef(new ExecutablePool(mem)); 317 } CC'ing Gavin, he is the allocator expert.

Adrien Nader

Comment 8 2010-08-20 13:16:10 PDT

In order to try Zoltan's suggestion, I updated to HEAD (from webkit-gtk 1.3.3, which is not a very old release) and the problem evolved: backtrace changed, both happens with and without JIT, but it still only happens on the laptop and not the quad-core. Another interesting point: I added 5GB swap on the laptop (from the 2GB of RAM) and jsc is now working properly. :-) Program received signal SIGSEGV, Segmentation fault. 0x00000000004e7010 in FixedVMPoolAllocator (this=0x9409c0, commonSize=16384, totalHeapSize=2147483648) at JavaScriptCore/jit/ExecutableAllocatorFixedVMPool.cpp:303 (gdb) bt #0 0x00000000004e7010 in FixedVMPoolAllocator (this=0x9409c0, commonSize=16384, totalHeapSize=2147483648) at JavaScriptCore/jit/ExecutableAllocatorFixedVMPool.cpp:303 #1 0x00000000004e5de6 in JSC::ExecutableAllocator::isValid (this=0x93fa18) at JavaScriptCore/jit/ExecutableAllocatorFixedVMPool.cpp:442 #2 0x0000000000472684 in JSC::ExecutableAllocator::ExecutableAllocator() () #3 0x00000000004714dd in JSGlobalData (this=0x93e240, globalDataType=JSC::JSGlobalData::Default, threadStackType=JSC::ThreadStackTypeLarge) at JavaScriptCore/runtime/JSGlobalData.cpp:150 #4 0x000000000047205c in JSC::JSGlobalData::create (type=JSC::ThreadStackTypeLarge) at JavaScriptCore/runtime/JSGlobalData.cpp:231 #5 0x000000000040d47d in main (argc=1, argv=0x7fffffffdde8) at JavaScriptCore/jsc.cpp:346

Maciej Piechotka

Comment 9 2010-08-26 06:56:10 PDT

I reproduced it without swap on Core 2 Duo (64 bit system) [4 GiB of memory + 0 GiB of swap]. ==4456== Process terminating with default action of signal 11 (SIGSEGV): dumping core ==4456== Access not within mapped region at address 0x27C ==4456== at 0x4C2876D: memcpy (mc_replace_strmem.c:482) ==4456== by 0x6C60525: JSC::JIT::privateCompileCTIMachineTrampolines(WTF::RefPtr<JSC::ExecutablePool>*, JSC::JSGlobalData*, JSC::TrampolineStructure*) (AssemblerBuffer.h:138) ==4456== by 0x6C8650F: JSC::JIT::compileCTIMachineTrampolines(JSC::JSGlobalData*, WTF::RefPtr<JSC::ExecutablePool>*, JSC::TrampolineStructure*) (JIT.h:223) ==4456== by 0x6D008E4: JSC::JSGlobalData::JSGlobalData(JSC::JSGlobalData::GlobalDataType, JSC::ThreadStackType) (JSGlobalData.cpp:152) ==4456== by 0x6D00EC2: JSC::JSGlobalData::create(JSC::ThreadStackType) (JSGlobalData.cpp:225) ==4456== by 0x6D00F01: JSC::JSGlobalData::createLeaked(JSC::ThreadStackType) (JSGlobalData.cpp:231) ==4456== by 0x6464641: WebCore::JSDOMWindowBase::commonJSGlobalData() (JSDOMWindowBase.cpp:160) ==4456== by 0x64A906B: WebCore::ScriptController::getAllWorlds(WTF::Vector<WebCore::DOMWrapperWorld*, 0ul>&) (ScriptController.cpp:187) ==4456== by 0x678F271: WebCore::FrameLoader::dispatchDidClearWindowObjectsInAllWorlds() (FrameLoader.cpp:3382) ==4456== by 0x678F306: WebCore::FrameLoader::receivedFirstData() (FrameLoader.cpp:668) ==4456== by 0x678459F: WebCore::DocumentWriter::setEncoding(WebCore::String const&, bool) (DocumentWriter.cpp:236) ==4456== by 0x6B7DCA0: WebKit::FrameLoaderClient::committedLoad(WebCore::DocumentLoader*, char const*, int) (FrameLoaderClientGtk.cpp:152) With swap on everything is OK [4 GiB of memory + 4 GiB of swap]. Swap is not uses and in total 2.2 GiB is used. Linux localhost 2.6.34-zen1-static #1 ZEN SMP PREEMPT Tue Aug 24 20:09:30 CEST 2010 x86_64 Intel(R) Core(TM)2 Duo CPU T9600 @ 2.80GHz GenuineIntel GNU/Linux

ojab

Comment 10 2010-11-14 05:02:34 PST

Looks like the same crash also happens for me on Athlon X2 II (Linux x86_64, glibc-2.11.2, no swap), only with JIT.

ojab

Comment 11 2010-11-14 05:05:41 PST

Created attachment 73848 [details] Segfault when running jsc without any argument (debug build) WebKit-trunk, git-svn-id: http://svn.webkit.org/repository/webkit/trunk@71981 268f45cc-cd09-0410-ab3c-d52691b4dbfc

ojab

Comment 12 2010-11-19 11:54:04 PST

git-svn-id: http://svn.webkit.org/repository/webkit/trunk@72416 268f45cc-cd09-0410-ab3c-d52691b4dbfc Still crashes.

Oliver Hunt

Comment 13 2010-11-19 12:13:51 PST

Is anyone able to repro this crash with their own build? If so can you see if VM_POOL_ASLR is disabled? If it is disabled try reducing the size of VM_POOL_SIZE in ExecutableAllocatorFixedVMPool.cpp to 1gig, and then 512Mb and see if that fixes things? I'm trying to determine exactly what part of this call is failing. Thanks.

ojab

Comment 14 2010-11-20 13:32:45 PST

OS isn't DARWIN, so I assume that VM_POOL_ASLR is disabled (don't know how to check). VM_POOL_SIZE = 1Gb/512Mb fixes the issue, jit build doesn't crashes.

Oliver Hunt

Comment 15 2010-11-20 20:32:33 PST

(In reply to comment #14) > OS isn't DARWIN, so I assume that VM_POOL_ASLR is disabled (don't know how to check). > VM_POOL_SIZE = 1Gb/512Mb fixes the issue, jit build doesn't crashes. Could you do me a favour, and on your platform see whether an allocation (via mmap) of 2GB succeeds in a trivial test program? Maybe there's an upper limit to the amount of memory your platform allows in a single alloc?

ojab

Comment 16 2010-11-21 12:57:49 PST

Unfortunately I don't know C and cannot write testcase to allocate 2Gb via mmap .__. Testcase with malloc() works without crashes. I have 2Gb ram total and if memory will be used (not only allocated) — program will be killed by OOM killer. WebKit with jit crashes without OOM killer participation.

ojab

Comment 17 2010-11-25 23:58:27 PST

Any additional info needed?

Zoltan Herczeg

Comment 18 2010-11-26 00:12:34 PST

(In reply to comment #16) > Unfortunately I don't know C and cannot write testcase to allocate 2Gb via mmap .__. Testcase with malloc() works without crashes. mmap_test.c: #include <stdio.h> #include <sys/mman.h> int main() { printf("%p\n", mmap(0, 2ul * 1024ul * 1024ul * 1024ul, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANON, -1, 0)); return 0; } Compiling: gcc mmap_test.c -o mmap_test

ojab

Comment 19 2010-11-26 01:15:39 PST

$ ./mmap 0xffffffffffffffff works fine.

Zoltan Herczeg

Comment 20 2010-11-26 01:31:35 PST

(In reply to comment #19) > $ ./mmap > 0xffffffffffffffff > > works fine. 0xffffffffffffffff (is -1) is MAP_FAILED (you can check it by a printf MAP_FAILED) Thus Oliver were right, and you are not allowed to alloc 2G. Could you try it with smaller sizes? (1G, 512M, etc.) until you get something back other than MAP_FAILED ?

ojab

Comment 21 2010-11-26 01:58:06 PST

ook.. (1024ul * 1024ul * 1024ul + 1024ul * 1024ul * 256ul ) in the `while true; do ./mmap && sleep 1; done` loop gives me output like: 0x7f7f21422000 0x7fe0ffac7000 0xffffffffffffffff 0x7f2f98c9a000 0x7f76b0a50000 0xffffffffffffffff 0x7f6444a30000 0x7f2da6a14000 0xffffffffffffffff With 1G I have normal addressess, but if I'll run memory-hungry task — I'll see the same sometimes-0xffffffffffffffff output. Looks like my system can only allocate amount of memory that is free at the moment.

ojab

Comment 22 2011-01-05 20:09:31 PST

git-svn-id: http://svn.webkit.org/repository/webkit/trunk@75066 268f45cc-cd09-0410-ab3c-d52691b4dbfc Still the same crash.

Gavin Barraclough

Comment 23 2011-01-06 01:10:36 PST

JSC really needs the OS to provide a mechanism to be able to reserve virtual memory address ranges (in wtf/PageReservation.h). As well as being used to allocate JIT code buffers, this is also used in the collector. To support WebKit on Linux without swap, we really need a way for this to work. PageReservation presently works on HAVE_MMAP platforms (including Linux) by simply mmapping a (potentially very large) region of memory, on the expectation that physical pages will be allocated on demand to the VM space. If the OS does not allow memory to be overcommitted this isn't going to work. I imagine the problem for someone to solve here is probably to find a way for the Linux kernel to reserve VM address ranges in a threadsafe fashion.

Gavin Barraclough

Comment 24 2011-01-12 11:56:54 PST

Interesting comment in redhat's bug tracker: https://bugzilla.redhat.com/show_bug.cgi?id=648319 Pete Zaitcev 2010-12-22 20:03:51 EST BTW, an easy workaround if you don't want to use my RPM from comment #31 is: echo 1 > /proc/sys/vm/overcommit_memory Well, or just buy a couple gigs of RAM. A more graceful failure for WebKit might be to read '/proc/sys/vm/overcommit_memory' on startup, and if this is not enabled then exit with an error message indicating that this is required.

Gavin Barraclough

Comment 25 2011-01-12 11:57:38 PST

*** Bug 51511 has been marked as a duplicate of this bug. ***

Geoffrey Garen

Comment 26 2011-01-12 13:32:47 PST

Wow -- so, by default, Linux will not commit VM on demand?

Adrien Nader

Comment 27 2011-01-12 14:05:36 PST

I haven't been able to provide a lot of feedback, mostly because of exams but what really leaves me wondering is that the JIT tries to get 4GB of memory. 4GB. It might not use it but 4GB is simply too much. What's actually the point? Do we really want that to happen? And why ask for that much if it's not going to be used? IIRC that also happens without even using javascript. Why does the JIT allocate a huge amount of memory without even requiring any memory actually?

Oliver Hunt

Comment 28 2011-01-12 14:09:31 PST

(In reply to comment #27) > I haven't been able to provide a lot of feedback, mostly because of exams but what really leaves me wondering is that the JIT tries to get 4GB of memory. 4GB. > > It might not use it but 4GB is simply too much. > > What's actually the point? Do we really want that to happen? And why ask for that much if it's not going to be used? > > IIRC that also happens without even using javascript. Why does the JIT allocate a huge amount of memory without even requiring any memory actually? It should only be requesting 2gig, and it's expecting the OS to merely reserve address space -- it's not expecting physical memory to be committed.

Adrien Nader

Comment 29 2011-01-12 14:16:36 PST

> It should only be requesting 2gig, and it's expecting the OS to merely reserve address space -- it's not expecting physical memory to be committed. 2GB is still a lot. And if it counts on the OS not commiting it, then it counts on the memory not being used. Why ask for it then? Quoting the linux kernel documentation for memory overcommit, > When this flag is 1, the kernel pretends there is always enough > memory until it actually runs out. I doubt that will ever become the default. It also leaves out a number of OSes that don't use overcommit (iirc, Solaris doesn't for instance). Why isn't it possible to request 64MB instead? What's the technical reason for asking that much?

Oliver Hunt

Comment 30 2011-01-12 14:25:08 PST

(In reply to comment #29) > > It should only be requesting 2gig, and it's expecting the OS to merely reserve address space -- it's not expecting physical memory to be committed. > > 2GB is still a lot. > > And if it counts on the OS not commiting it, then it counts on the memory not being used. Why ask for it then? > > > Quoting the linux kernel documentation for memory overcommit, > > When this flag is 1, the kernel pretends there is always enough > > memory until it actually runs out. > > I doubt that will ever become the default. > It also leaves out a number of OSes that don't use overcommit (iirc, Solaris doesn't for instance). > > Why isn't it possible to request 64MB instead? What's the technical reason for asking that much? We allocate 2gig up front and do all allocations of executable memory from that region as we need to be able to guarantee that all executable allocations are within 2gig of each other - it would be a large amount of work (and a significant increase in code size) to support the arbitrary jumps to anywhere in the address space that would otherwise be necessary. Surely Linux must have some mechanism for reserving an arbitrarily large amount of address space, and then commit that lazily?

Xan Lopez

Comment 31 2011-01-12 14:26:26 PST

Created attachment 78732 [details] overcommit.diff Patch for this issue.

Oliver Hunt

Comment 32 2011-01-12 14:52:27 PST

Comment on attachment 78732 [details] overcommit.diff i think 32mb will be far too little to be useful -- a simple test would be for you to just try using midori with a jsc build with vmPoolSizeGeneric == 32mb Also i think the logic to determine what size the allocation should be would be better in a separate function.

Xan Lopez

Comment 33 2011-01-12 15:02:11 PST

(In reply to comment #32) > (From update of attachment 78732 [details]) > i think 32mb will be far too little to be useful -- a simple test would be for you to just try using midori with a jsc build with vmPoolSizeGeneric == 32mb I did try running a browser with those values for some minutes and things didn't explode right away, but you are right that they are fairly conservative. Do you want to make them bigger in general or do you want a triplet like generic/conservative desktop/conservative embedded. If the answer is the latter, how will we know which one to fallback in linux when overcommit is not on? > > Also i think the logic to determine what size the allocation should be would be better in a separate function.

Gavin Barraclough

Comment 34 2011-01-12 15:03:41 PST

Comment on attachment 78732 [details] overcommit.diff Looks great xan!

Gavin Barraclough

Comment 35 2011-01-12 15:07:20 PST

(In reply to comment #29) > > It should only be requesting 2gig, and it's expecting the OS to merely reserve address space -- it's not expecting physical memory to be committed. > > 2GB is still a lot. For 64-bit OSes that allow more flexible control over the VM space , 2GB is less than 1% of 1% of available VM. This is not a lot. ;-) For OSes that insist on early commit it clearly doesn't seem sensible to make such a large request. (In reply to comment #30) > Surely Linux must have some mechanism for reserving an arbitrarily large amount of address space, and then commit that lazily? This would be an improvement over the current fix, if we can find a way to do so.

Gavin Barraclough

Comment 36 2011-01-12 15:08:58 PST

Ooops, my r+ crossed with Oliver's comments, (In reply to comment #32) > (From update of attachment 78732 [details]) > i think 32mb will be far too little to be useful -- a simple test would be for you to just try using midori with a jsc build with vmPoolSizeGeneric == 32mb Not that r74454 should help prevent this from causing problems. > Also i think the logic to determine what size the allocation should be would be better in a separate function. ^ olliej's right, you should probably do this.

Xan Lopez

Comment 37 2011-01-12 17:04:17 PST

Created attachment 78761 [details] overcommit.diff New patch moving the code that messes with the values to a function. No change in the values because: - We are still discussing that. - It probably even makes sense to separate these two changes. - Even if only this goes in, it's already an improvement in general for many people in 64bit Linux.

Gavin Barraclough

Comment 38 2011-01-12 18:44:21 PST

Comment on attachment 78761 [details] overcommit.diff Looks great.

Xan Lopez

Comment 39 2011-01-13 04:03:21 PST

Comment on attachment 78761 [details] overcommit.diff Landed in http://trac.webkit.org/changeset/75709 I'll open a new bug in case we want to improve the VM values for !overcommit in Linux.

Xan Lopez

Comment 40 2011-01-13 04:03:41 PST

Closing.

Gavin Barraclough

Comment 41 2011-01-28 17:09:20 PST

See changes in https://bugs.webkit.org/show_bug.cgi?id=53352

Gavin Barraclough

Comment 42 2011-01-31 11:29:52 PST

Reopening due to bug #53352. The upshot of this bug is, the fix landed here wouldn't work anyway (without overcommit you'll allocate the smaller sized buffer, then crash after a short period of browsing once the JIT buffer became nicely fragmented). Please see comment in bug #53352, options are to not support this configuration, always use the reduced size (and break some stress tests), or reimplement a dynamic/static switch based on whether overcommit is available. If we need a switching solution, one option might be to always use 1GB tables, to just change the amount we allocate, and then to add a method to the AllocateTableDirectory to mark ranges of the table as unavailable.

Eric Seidel (no email)

Comment 43 2011-01-31 16:07:42 PST

Comment on attachment 78732 [details] overcommit.diff Cleared Gavin Barraclough's review+ from obsolete attachment 78732 [details] so that this bug does not appear in http://webkit.org/pending-commit.

Martin Robinson

Comment 44 2011-02-03 12:27:21 PST

*** Bug 53684 has been marked as a duplicate of this bug. ***

Xan Lopez

Comment 45 2011-05-20 07:53:24 PDT

Hey, good news: turns out Linux has a flag for mmap that is perfect for this, MAP_NORESERVE. From man proc(5): /proc/sys/vm/overcommit_memory This file contains the kernel virtual memory accounting mode. Values are: 0: heuristic overcommit (this is the default) 1: always overcommit, never check 2: always check, never overcommit In mode 0, calls of mmap(2) with MAP_NORESERVE are not checked, and the default check is very weak, leading to the risk of getting a process "OOM-killed". Under Linux 2.4 any nonzero value implies mode 1. In mode 2 (available since Linux 2.6), the total virtual address space on the system is limited to (SS + RAM*(r/100)), where SS is the size of the swap space, and RAM is the size of the physical memory, and r is the contents of the file /proc/sys/vm/overcommit_ratio. -- 0 is the default on Linux, so I think if we just add a one-line patch in the code adding this flag by default things should work OK. I'll test this and report back.

Xan Lopez

Comment 46 2011-05-20 08:12:55 PDT

Created attachment 94221 [details] Patch

Oliver Hunt

Comment 47 2011-05-20 08:50:40 PDT

Comment on attachment 94221 [details] Patch r=me

Xan Lopez

Comment 48 2011-05-20 08:55:27 PDT

Comment on attachment 94221 [details] Patch Clearing flags on attachment: 94221 Committed r86957: <http://trac.webkit.org/changeset/86957>

Xan Lopez

Comment 49 2011-05-20 08:55:39 PDT

All reviewed patches have been landed. Closing bug.

Ademar Reis

Comment 50 2011-05-20 10:10:07 PDT

Revision r86957 cherry-picked into qtwebkit-2.2 with commit ac3c389 <http://gitorious.org/webkit/qtwebkit/commit/ac3c389>

Gavin Barraclough

Comment 51 2011-05-23 14:24:29 PDT

Nice!

Alexey Proskuryakov

Comment 52 2011-08-05 12:07:02 PDT