repo/checkout: Cache lookups of dirmeta objects

I was reading a strace the other day and noticed we were loading the same
`.dirmeta` object many times. Unlike the other object types, `.dirmeta` objects
don't accumulate much over time; there are only so many directory metadata types.
(Without SELinux involved it'd probably be 5-6 I'd guess offhand).

For `fedora-atomic/25/x86_64/docker-host` there are currently 34 `.dirmeta` in
the tree.

But how many times during a checkout did we load those 34 dirmeta objects?
With a quick strace:

```
$ strace -s 2048 -f -o strace.log ostree --repo=repo-build checkout -U fedora-atomic/25/x86_64/docker-host host-test-checkout
$ grep dirmeta strace.log | wc -l
7165
```

After, as you'd expect, we just loaded `34` from disk.  We do
6 system calls (`openat+fstat+fstat+read+read+close`) per dirmeta,
so we dropped a total of 42780 system calls - which is about 20% of the total
system calls made.

`perf record` tells me that we're spending ~40 of our time in the kernel during
a checkout, so reducing syscall traffic helps. Though most of that appears to be
in the VFS and XFS layers for `linkat` (which isn't surprising).

So how much did perf improve? Well, on my workstation, I get a lot of
fluctuation in timing, sometimes by 30%, so this was well within the noise. But
it's well worth speeding up checkout, and I think this optimization will shine
more as we improve performance elsewhere.

Closes: #795
Approved by: jlebon
This commit is contained in:
Colin Walters authored and Atomic Bot committed 2017-04-25 13:40:53 +00:00
1 parent 4fc65b808a
commit b7afe91e21
3 files changed
+85

No files matched your search

+21
View File
@@ -92,6 +92,9 @@ struct OstreeRepo {
OstreeRepoTransactionStats txn_stats;
GMutex cache_lock;
guint dirmeta_cache_refcount;
/* char * checksum → GVariant * for dirmeta objects, used in the checkout path */
GHashTable *dirmeta_cache;
gboolean inited;
gboolean writable;
@@ -126,6 +129,24 @@ typedef struct {
char checksum[OSTREE_SHA256_STRING_LEN+1];
} OstreeDevIno;
/* A MemoryCacheRef is an in-memory cache of objects (currently just DIRMETA). This can
* be used when performing an operation that traverses a repository in someway. Currently,
* the primary use case is ostree_repo_checkout_at() avoiding lots of duplicate dirmeta
* lookups.
*/
typedef struct {
OstreeRepo *repo;
} OstreeRepoMemoryCacheRef;
void
_ostree_repo_memory_cache_ref_init (OstreeRepoMemoryCacheRef *state,
OstreeRepo *repo);
void
_ostree_repo_memory_cache_ref_destroy (OstreeRepoMemoryCacheRef *state);
G_DEFINE_AUTO_CLEANUP_CLEAR_FUNC(OstreeRepoMemoryCacheRef, _ostree_repo_memory_cache_ref_destroy);
#define OSTREE_REPO_TMPDIR_STAGING "staging-"
#define OSTREE_REPO_TMPDIR_FETCHER "fetcher-"