repo/checkout: Cache lookups of dirmeta objects
I was reading a strace the other day and noticed we were loading the same `.dirmeta` object many times. Unlike the other object types, `.dirmeta` objects don't accumulate much over time; there are only so many directory metadata types. (Without SELinux involved it'd probably be 5-6 I'd guess offhand). For `fedora-atomic/25/x86_64/docker-host` there are currently 34 `.dirmeta` in the tree. But how many times during a checkout did we load those 34 dirmeta objects? With a quick strace: ``` $ strace -s 2048 -f -o strace.log ostree --repo=repo-build checkout -U fedora-atomic/25/x86_64/docker-host host-test-checkout $ grep dirmeta strace.log | wc -l 7165 ``` After, as you'd expect, we just loaded `34` from disk. We do 6 system calls (`openat+fstat+fstat+read+read+close`) per dirmeta, so we dropped a total of 42780 system calls - which is about 20% of the total system calls made. `perf record` tells me that we're spending ~40 of our time in the kernel during a checkout, so reducing syscall traffic helps. Though most of that appears to be in the VFS and XFS layers for `linkat` (which isn't surprising). So how much did perf improve? Well, on my workstation, I get a lot of fluctuation in timing, sometimes by 30%, so this was well within the noise. But it's well worth speeding up checkout, and I think this optimization will shine more as we improve performance elsewhere. Closes: #795 Approved by: jlebon
This commit is contained in:
1 parent
4fc65b808a
commit
b7afe91e21
3 files changed
+85
No files matched your search
@@ -92,6 +92,9 @@ struct OstreeRepo {
|
||||
OstreeRepoTransactionStats txn_stats;
|
||||
|
||||
GMutex cache_lock;
|
||||
guint dirmeta_cache_refcount;
|
||||
/* char * checksum → GVariant * for dirmeta objects, used in the checkout path */
|
||||
GHashTable *dirmeta_cache;
|
||||
|
||||
gboolean inited;
|
||||
gboolean writable;
|
||||
@@ -126,6 +129,24 @@ typedef struct {
|
||||
char checksum[OSTREE_SHA256_STRING_LEN+1];
|
||||
} OstreeDevIno;
|
||||
|
||||
/* A MemoryCacheRef is an in-memory cache of objects (currently just DIRMETA). This can
|
||||
* be used when performing an operation that traverses a repository in someway. Currently,
|
||||
* the primary use case is ostree_repo_checkout_at() avoiding lots of duplicate dirmeta
|
||||
* lookups.
|
||||
*/
|
||||
typedef struct {
|
||||
OstreeRepo *repo;
|
||||
} OstreeRepoMemoryCacheRef;
|
||||
|
||||
|
||||
void
|
||||
_ostree_repo_memory_cache_ref_init (OstreeRepoMemoryCacheRef *state,
|
||||
OstreeRepo *repo);
|
||||
|
||||
void
|
||||
_ostree_repo_memory_cache_ref_destroy (OstreeRepoMemoryCacheRef *state);
|
||||
G_DEFINE_AUTO_CLEANUP_CLEAR_FUNC(OstreeRepoMemoryCacheRef, _ostree_repo_memory_cache_ref_destroy);
|
||||
|
||||
#define OSTREE_REPO_TMPDIR_STAGING "staging-"
|
||||
#define OSTREE_REPO_TMPDIR_FETCHER "fetcher-"
|
||||
|
||||
|
||||
Reference in new issue
Block a user