Best transcoding software?

[Gieferg]

I have a problem and I am trying to solve that for few days now.
I have a 30GB M2TS file encoded with MPEG-2 codec, 1080i, 25fps. Quality is good enough, but I need to burn it on blu-ray disc (as standard blu-ray file) and it needs to go under 20GB. Usually I use MultiAVCHD for that and it worked fine until now. But for some reason, with this particular file, it creates artifacts in the form of ugly aliasing on some straight lines on some of the shots.

So I tried DVDFab (conversion to m2TS), the output looked fine, but for some reason doesn't work on Blu-ray player (which just freezed every time I tried to play this). Then I tried DVDFab but just Blu-ray copy BD50 to BD25 - and framerate was messed up.

Then I tried Avisynth/MeGui - file was ok on my computer but didn't play properly on Blu-ray player (ugly artifacts, picture literally falls apart into pixelated mess)

So I tried DaVinci Resolve but it looks like it doesnt really have options for good compression - so 19GB file works fine, there is no aliasing, but compression looks really ugly especially on shots with the light directly on the camera lens.

What else can I try?

EDIT:

Trying to convert it with Sony Vegas, didn't try that earlier because Vegas didnt want to open the file, but I've managed to handle that.

 

[unfair]

ffmpeg is my go-to for any transcoding, you can adjust anything you want from quality to file type, but it would be a bit of trial and error to figure out what quality equates to the size you want. Perhaps there are some online calculators since it's a widely used tool, not sure, never looked.

This is an example command I use when I have to convert an mp4/mkv to aac audio but want to leave the video unchanged:
C:\ffmpeg\bin\ffmpeg.exe -i "YOURMOVIENAME.mkv" -c:v copy -ac 6 -ar 48000 -ab 1536k -c:a aac "NEWMOVIENAME-dol.mp4"

For your issue you'll probably want to do the opposite, but that command should give you a starting point in the manuals to look at some of the basic commands. It's one of those tools that has a learning curve but once you get your commands figured out it's invaluable. I think you can find GUIs for it too but for transcoding you really don't need one.

 

[krausfadr]

I find when I have a problem file, a re-encode with Handbrake solves it.

 

[Gieferg]

I will look into it, but I remember I tried it once and wasn't too happy with it.

This is an example command

When it comes to programs using command lines / scripts, I am completely lost (really hate Avisynth).

Meanwhile, Vegas gave me a file which doesnt seem to have aliasing when watched on my computer and shows it on Blu-ray player (no matter what settings I choose)... what the hell?

 

[Gieferg]

I tried DVDFab again, changed level in TS_Muxer from 4.1 to 4.0 and it works!

By the way, looks like DVDFab is able to do WAY better compression at the same bitrate than Resolve which looks really pathethic in comparison.

 

[krausfadr]

Yeah Resolve and most NLE’s are terrible at H264 and H265.

 

[Gieferg]

This is quite surprising, given the amount of people that use Resolve. Or maybe there is some other way to export files from it? Because after what I've seen, I consider it to be useless for me.

 

[krausfadr]

You can export from NLE’s to a mastering format like Prores or Cineform. Then convert that using a good freeware encoder that does the job correctly. NLE’s can double the overall bitrate of H264 and H265 and what is the reward for this… blocky artifacts in dark scenes.

 

[damian101]

For me, Av1an, because it supports chunked encoding, directly uses the system's encoder binaries and allows vapoursynth scripts as input.

 

[TriggeredPuppy]

I know this thread is a little old but wanted to throw out Shutter Encoder as another option. It has pretty wide file support and is more of a frontend for other free / command line video encoding software applications. In my experience it handles most anything I've thrown at it with very good results. And would solve the preference to avoid command line programs!