php - How do I efficiently load data from MySQL into a cache?
Working with a large database and there is a value that needs to be cached. The value is computed from a nested MySQL query -> load result into PHP object -> and then cached. The number of values that need to be cached is n^2 and n ~= 40000. To create the cache for all n, it would take days and it's too slow.
So naturally, I think about running parallel processes using PHP multi_curl for example. But then I realized that the CPU load is not exactly negligible either. The
load result into PHP object ends up taking a fair amount of CPU. And the upper number parallel processes we can run before server can't handle it anymore is just not enough.
Aside from upgrading the server, are there any other suggestions for how I can run this one-time script to populate the cache for O(40000^2) values in a reasonable amount of time?
The SQL query in question is something along the lines of:
SELECT value FROM table_a WHERE id = $x AND value IN (SELECT value FROM table_a WHERE id = $y)