AFAIK this is a common problem in rendering, be it CSS or games: how do you not render something that's not currently visible on screen. There are all sorts of tricks to calculate the "render only this" set as quickly as possible.