Today was having fun with filesystem caches day.
Firefox warm start: ~820ms
Firefox cold start: ~11200ms
Now let’s manually preread some things (via tar chf /dev/null), after purging:
preread app dir/profile dir, start: ~7800-8500ms
preread system libs: ~7800ms-8600ms
preread both: ~7800ms-8600ms
So somewhere we’re missing 7000ms worth of IO. Staring at Shark, dtrace, etc. to try to figure out where. Fonts maybe? Something else? Whatever it is it’s not showing up in dtruss, which is weird.
purge, launch another app, start: ~5100ms
I’m not sure if my attempt to prewarm via tar/cat is having the desired effect, so I wrote a quick tool that will mmap a file with PROT_EXEC and then read from every page in it. The numbers are fairly similar, but I can tell that we’re definitely reading parts of various libraries/files, because the time to prewarm after a purge is greater than the time to do so after a purge and a Firefox start.
Still, I want to figure out what the kernel’s filesystem caches look like during a “warm” Firefox start, and how to get in that state without launching Firefox itself. This will give us an idea of what IO we need to remove and/or optimize.
Today’s bugs filed: bug 506812 – get rid of remaining FSRef uses on MacOS X, bug 506814 – get rid of/change consumers of [SG]etPersistentDescriptor, and bug 506841 – nsChromeRegistry’s CheckForNewChrome runs on every launch. They’re related, though the first one in particular should impact all the rest… CheckForNewChrome, besides being unnecessary to do every launch, is slow mainly because it enumerates the chrome directories, which involves creating FSRefs for every file.
When I run “lsof -c firefox-bin” on a new profile, I see..
app dir: 5 chrome jars, 17 dylibs
/Library: 1 caches, 19 fonts, 2 plugins (flash, quicktime)
/System/Library: 1 caches, 4 encoding dylibs, 4 fonts, ~15 framework related (carbon, core audio, core uiquicktime)
profile dir: ~10 sqlite/db
user/Library: 4 caches, 1 sqlite, 2 fonts
I suppose it might be more useful if it was a lsof -c firefox-bin at various points during startup, but that might be useful in figuring out what else might be loaded.
I vaguely remember that several sqlite files still used the default 1K pagesize, instead of the normal 4K. Not places.sqlite, but cookies.sqlite etc … I’ll try to look tonight if I can make a list of them. I saw the 1K reads in a Process Monitor logfile, but I don’t know if they would tax the startup a lot.
It was mentioned somewhere in a bug that we should force them all to use 4K. Note that if we would change the pagesize, then people with old profiles would still use the incorrect pagesizes. A workaround (aside form recreating the databases) would be to create a new profile.
You can get a complete list of what’s in cache by stracing the program first
Taras, yeah, as I mentioned I have a full trace; it’s not enough.
Ed, yep, I compared that with the trace that I have. Mostly the same files. There’s something else going on, and I’m not sure what, will keep looking.
You might want to look at ‘cinfo’ – http://ds9a.nl/cinfo
It has bitrotted slightly, but it will tell you which pages of a file are in the page cache (on Linux, and perhaps Solaris too).