I have wanted to write a similar thing for a while now. I would do it
with processes:
1) Create a server to manage the ets table (eventually you would want
this distributed), which handles the following messages:
- New: create a new process and insert in ets with key
- Locate: see if a key leads to an existing process
- Reap: remove key and process from ets
2) Each process has a receive which does the following:
- GetData: returns the state and loops
- after LRU_Timeout: calls ets manager with Reap(SelfKey) then dies
Creating a new process causes it to read data from DB, construct
corresponding internal data representation and then wait for callers to
request the reformatted data. Once it is idle for a long enough period
of time it dies. If there is a request prior to expiration, the
LRU_Timeout starts over so every request is a new lease on life.
Variations:
a) Vary the LRU_Timeout based on the difficulty of restart.
b) Add supervisors to organize processes into hierarchies of related
data (or just link the related processes as they are created by
cascading spawns). Kill the supervisor (or one of the networked
processes) to remove all dependencies from memory.
c) Use a reaper process which monitors total memory consumption. Have
it broadcast a message to hasten the demise of any process which deems
it can time out early in a memory pinch.
jay