My real goal is to find a chain of programs and settings that will do an acceptable job of coloring and upscaling, require minimal manual work, and be able to mass reproduce the results for whole black and white seasons (and eventually fan edit them). Frankly I'm nowhere near achieving that either, this one image took a lot of manual work and I'm still poking at it making little improvements and changes.

The color in this started with Photoshop's new neural filters, there's one filter to colorize automatically. I found it a bit bland so I changed the colors a lot after the fact, but that blandness I think will be at least consistent. As in, I think you could feed in a ton of frames of footage and the quality and temperature of the colors will remain stable.

This means there's a starting point, software that will (possibly, testing pending) work for semi-automating colorization of video without flickering badly, so at a later point colors can be enhanced or altered in a relatively small quantity keyframes and the keyframes' style can be transferred to all video frames either with Ebsynth (which isn't magic but should work with color adjustments perfectly) or possibly with Photoshop's new style transfer tool (which I haven't tested yet). That's a lot of maybes, but there's hope.

I'm also counting this as separate tests, there's the upscale, and there's the colorization. One might work better than the other and prove more feasible.

By the time the workflow is ironed out and the output looks good enough to me, the first and second Doctor Blu rays may have already released and would probably give the cleanest starting point for source footage. This experiment will be slow until more software magic shows up.