AI Enhanced Videos

[luckgat2d]

Has anyone tried Topaz on the newest MacBook Pros? Or should not I even bother without a desktop?

[jbc313]

Works on my MacBook Air with the new M2 chip. I asked Topaz support and they said that making sure the software performs well on new Apple silicon is something they are focusing on.

[DarkRaven671]

Terms like "works" and "performs well" are very relative. Of course they are saying that this is something they are focusing on, because this is a part of their potential customer base.

However, context is needed to properly gauge how much actual performance can be expected.

The M2 chip is not a bad SoC for a mobile device. But "being good for a mobile device" and "being good for upscaling" are two entirely different things. Topaz will "work" on most systems, there's no reason why it wouldn't work. But it will only perform on a powerful GPU.

How powerful is the GPU of the M2 SoC? Well, according to its specs, it has a raw computing power of 3.6 Teraflops per second (TFLOPS). A Flop is a floating point operation and how many of those a chip can perform creates an index to describe and compare the computing capabilities of a chip. A Teraflop is 1.000.000.000.000 FLOPS.

Is 3.6 TFLOPS a lot? For the integrated GPU in a SoC, yeah, it's really good. But in the overall grand scheme of things, no, it isn't.
For comparison:

The latest nVidia GPU, the RTX 4090, performs 73 TFLOPS, which is 20 times as much.

A previous generation GPU, like the RTX 3080 Ti, which is the one I'm using and it isn't even the biggest one in the family, performs 28 TFLOPS, which is 7.7 times as much.

How far down do we have to go to find something with similar raw power? The RTX 3050 is the smallest offering from nVidia among the RTX 3000 family for desktop. Almost 8 TFLOPS, still nearly double the power of the M2 and this is a GPU that most people would say isn't really a good choice for upscaling. The mobile version of the RTX 3050 for use in a laptop performs 2.9 TFLOPS, so we're there.

And I'm not even considering that, because of temperature constraints, the M2 chip will not constantly run at full power.

So, yeah, it will surely work and performance is going to be ok if you're not processing too many videos. But don't expect too much, I wouldn't recommend it for more than the occasional upscale and laptops in general aren't necessarily the best choice for this, because many are known to develop temperature issues when running under high load for an extended period of time, which is also the case here.

[ShagBuddy]

Just adding some info to this nice thread... For upscales I use Topaz and then I do color corrections using StaxRip. I made a few posts showing how, along with some examples in this thread. I personally feel as though fixing the color and contrast of older scenes (which are usually under-saturated, washed out, tinted some crazy color etc.) makes a HUGE difference. I understand that there are purists who prefer to keep things as close to original as they can, but I prefer to make it look right.

I recently ran across this free app for encoding and tweaking files as well. it has a different approach to tweaking the video presentation with a simple GUI.

http://shutterencoder.com/

As far as hardware, I recently rebuilt my workstation (I do that every few years). The system I used before the rebuild:

AMD 3700X CPU
AMD 5700 XT GPU

Now currently running:

AMD 7900X CPU
AMD 7900XTX GPU

The difference is BIG. About 300% faster with everything.

You don't have to have super crazy hardware though. When I was processing videos with the 5700 XT I would setup some to run and just let it run overnight and let it keep running until I needed my PC for something else .

[kamaffer]

Originally Posted by liddokun420x

If there is anybody reading this thread feel free to comment. Even if you're a noob.

i see youve posted a lot for hybrid/stax but not a whole lot of help for avisynth+, could you point me in the direction of some helpful guides/tutorials for after having everything set up? or a list of functions/commands(?) that you use commonly with avisynth? thanks

[DarkRaven671]

I have a tutorial in the works and that will include AviSynth+. I wanted to have it finished last year, actually, but because of time constraints, it's a rather slow process. It's coming though, eventually.

In the meantime, here's a template of the (very simple) script I use:

SetFilterMTMode ("QTGMC", 2)
a=LWLibavAudioSource(source="<yourfile>", stream_index=-1, cache=true, av_sync=true, rate=0, decoder="")
v=LWLibavVideoSource("<yourfile>")
AudioDubEx(v,a)
AssumeTFF()
QTGMC(preset="Slower")
BilinearResize(768,576)
Prefetch(<threads>)
Trim(<firstframe>,<lastframe>)

This is specifically to use QTGMC to deinterlace DVD video and cut the scene I want to then save it as a high quality video file and/or use it as a base for upscaling. I don't have time to explain the details right now, but I'll get back to it later.

edit:
Ok, so, what it does is this:

SetFilterMTMode ("QTGMC", 2)

Sets the multithreading mode for QTGMC, 2 is the generic and compatible one, 1 and 3 are for specific use cases. 2 has the highest memory (RAM) usage.

a=LWLibavAudioSource(source="<yourfile>", stream_index=-1, cache=true, av_sync=true, rate=0, decoder="")
v=LWLibavVideoSource("<yourfile>")
AudioDubEx(v,a)

Opens your input video file. Audio and video are treated separately and are then combined again into one. Other methods to open your source exist, for example FFMPEGSource2. That one occasionally gave me trouble though, so I decided to not use it.

QTGMC(preset="Slower")

QTGMC has a number of presets available, see its documentation for all of them. You can set them individually or go with a preset. Unless there's a specific reason to configure it individually, the presets are generally the best solution for many people. I think "Slower" is the best compromise between quality and speed.

BilinearResize(768,576)

This one is a matter of personal taste and the given values are for PAL DVDs. NTSC DVDs would require (720,540). This adjusts the aspect ratio slightly to better match screens that aren't analog TVs. It's optional and I used to be split about it, but I've since made the decision that I like this better.
If you're not doing it, you will work with video in 720x576 for PAL or 720x480 for NTSC.

Prefetch(<threads>)

This needs a number and that number should be slightly lower than the amount of threads your CPU can process at the same time. Higher number = more performance, but also more CPU load, up to the point where your CPU would run at 100%, at this point further increasing it will have no effect or will actually make processing slower.
You have to try what works best for. I run a 24-Core CPU (48 threads with Hyperthreading) and found that 32 to 40 is the sweet spot. I'd suggest to start with 75% of the amount of threads your CPU can process and then go from there, maybe you might want it higher or lower. It depends on the CPU and whether you're doing something else while processing video.

Trim(<firstframe>,<lastframe>)

This cuts the input at the given frames. This is useful when you want a specific scene from a DVD. You could also cut it later, but doing this here means less processing and less storage required.

That's it, this is a pretty simple starting point which allows you to make QTGMC files which can then be encoded and used on their own, or upscaled prior to the final encoding.

[kamaffer]

do you resize in avisynth/virtualdub or use something like topaz just to upscale?

[DarkRaven671]

AviSynth offers several, non-AI ways of resizing, sharpening and so on. BilinearResize, which I use, is one of them. There's also an AI upscaling plugin, but I think it's not under active development anymore, the last version is dated mid-2021.

I've never used any of them though to significantly increase a video's resolution. Not sure how the results would look like. Until now I've exclusively used Topaz for upscaling.

[monte_christo]

Are there websites that will use AI to enhance and upgrade a video (that you would upload obviously)? That seems like a task that would scale well.