Here’s an example of a similar attempt: Bunch of LEDs. Not quite up to the level you’re asking about, but same order of magnitude. The power considerations remain as well.
FPS remains limited when syncing, but a single ESP32 is likely good for what you’ve described.
If you’re thinking of a 2D display type setup, that may have other considerations.