Building some fancy "drill press table" seems to be a popular "upgrade" for a drill press. But personally, I much prefer to use the drill press as is.
I just put a piece of hardwood loosely on the table. It keeps moving around, so I have a fresh piece of wood to drill into most times. Eventually, that piece of wood resembles a moonscape, but its still a lot more convenient than having to constantly replace the "insert" in a drill press table. I don't have to make sure my "insert" is exactly the same height as the table.
Some drilling operations need to be done on the edge of the table. I seem to hit these quite often, so a big drill press table would really get in the way.
I also move the drill press's own table up and down a lot. Again, some big drill press table attached to the small table would get in the way. Even if you extend the crank, the bulk of a larger table makes it more cumbersome to raise and lower.
Using the drill press can involve a lot of force. Here, I'm using a bathroom scale to measure the force of drilling a 1" hole in the end of a piece of birch with a Forstner bit. The scale reads 160 pounds (about 72 kg), yet I'm not even pushing very hard on the handle.
I get many suggestions that I should write an article about making a drill press using a hand drill, but good quality hand drills can cost more than a cheap drill press, and a cheap drill press is still better than a homemade one. So yes, you could build a drill press, but it would cost more and work worse than just buying one. I also came to that same conclusion when I experimented with building a hollow chisel mortiser. A similar conclusion by "Wooden Tool Man" (Wayne McLellan) realizing this as well in this video
Another justification for a drill press table is having a fence, which can be useful when you need to drill a hole in the same place in a lot of pieces of wood. But, personally, I prefer to just score the wood and punch it with an awl. The point of a drill will often wobble a bit, but if you make a divot where the hole should be, then the drill will naturally follow that, even if if you just eyeball where you put the workpiece.
But if I do need a drill press fence, I can always clamp a strip of wood to the table as a temporary fence.
I have to say though, I don't remember last time I used a fence on the drill press, but if you need to drill 50 parts the same way, and precision isn't critical, a drill press fence can be handy.
Come to think of it, I did use sort of a fence when I made this snake cube puzzle, although that fence consisted just of a piece of wood with a corner cut out of it. Handier than an actual fence. And I could have used a fence when I made the marble blocks, but I used my horizontal boring machine instead. The horizontal boring machine is more rigid, so it's more accurate.