php - How do I efficiently load data from MySQL into a cache?

81

Working with a large database and there is a value that needs to be cached. The value is computed from a nested MySQL query -> load result into PHP object -> and then cached. The number of values that need to be cached is n^2 and n ~= 40000. To create the cache for all n, it would take days and it's too slow.

So naturally, I think about running parallel processes using PHP multi_curl for example. But then I realized that the CPU load is not exactly negligible either. Theload result into PHP object ends up taking a fair amount of CPU. And the upper number parallel processes we can run before server can't handle it anymore is just not enough.

Aside from upgrading the server, are there any other suggestions for how I can run this one-time script to populate the cache for O(40000^2) values in a reasonable amount of time?

The SQL query in question is something along the lines of:

SELECT value FROM table_a WHERE id = $x
AND value IN
(SELECT value FROM table_a WHERE id = $y)

People are also looking for solutions to the problem: php - Simple IF/ELSEIF not comparing properly

Source

Didn't find the answer?

Our community is visited by hundreds of web development professionals every day. Ask your question and get a quick answer for free.

Ask a Question

Write quick answer

Do you know the answer to this question? Write a quick response to it. With your help, we will make our community stronger.

Similar questions

Find the answer in similar questions on our website.