xBRZ filter

[ericbazinga]

It's possible to pre-filter or even replace textures on some basis, but just remember that because DraStic uses a software renderer the expense for using higher resolution textures will be eaten on the CPU. This is contrary to PPSSPP which uses an OpenGL ES-based renderer. This also wouldn't have an impact on graphics rendered by the 2D engine.

In my opinion, if you're playing a predominantly 2D games you probably aren't benefitting much from the hires mode anyway.

I agree on the xRBZ idea, as it would fix the 2D-graphics-look-worse-by-comparison-on-hi-res problem. Yeah hi-res plus xRBZ would basically be an HD dream team.

[Hawntah]

How about adding an xBRZ filter applied to the whole screen, like the Scale2X and HQ2X filters already in DraStic? And perhaps some 3x magnification filters while you're at it? For those of us that play 2D games as well...

[Exophase]

How about adding an xBRZ filter applied to the whole screen, like the Scale2X and HQ2X filters already in DraStic? And perhaps some 3x magnification filters while you're at it? For those of us that play 2D games as well...

That's probably more realistic than an internal texture filter option. There might be some developments in the shader department in a later release.

[Fefo]

Whoa, thanks for those who posted the difference between xBRz and hqx. Tested it on Snes9X and it's far better with much less sttutering, an annoyance I have to deal.

I'm in for at least a full screen xBRz. Internal filtering would be optimal, but at least this filter results in a less blurry image.

[nintynuts]

I would be very happy with xBRZ implemented as a whole screen filter (as that was what my original post was about before High-Res was implemented).

However, my point with the internal filtering was that games like Pokemon use character sprites on planes in a 3D world. These would benefit greatly from xBRZ texture (sprite) upscaling.
Implementing a whole screen filter will not help with any game as long as the High-Res mode is on, because the number of pixels it has to work with is quadrupled.

Contiguous colours represent the current pixels, the actual grid (white) is the device pixels.

Filter vs Resolution.png (4.66 KiB) Viewed 17107 times

I have been playing Ace Attorney games, which are predominantly 2D, but have 3D object examinations for example.
I don't like having to compromise with upscaled 2D with poor 3D or high-Res 3D with blocky 2D,
firstly because going into the settings is a bit annoying (could it be put on the quick action buttons along with cycling filters?),
but also a single implementation of the upscaling seems to make more sense.

I wasn't aware the 3D rendering was entirely software implemented, I would be interested in knowing more why that would be more costly.

[huckleberrypie]

It is possible to render things through the GPU, but Exo chose not to do so due to Nintendo doing what they do best i.e. use non-standard, unorthodox hardware which is basically a pain to implement.

[nintynuts]

It is possible to render things through the GPU, but Exo chose not to do so due to Nintendo doing what they do best i.e. use non-standard, unorthodox hardware which is basically a pain to implement.

Sorry, I wasn't clear. I understand why emulators often have software implementations of the GPU, and that running on a CPU is generally less efficient than on the GPU.
What I was wondering was, what is it about it being software rendered that will be more costly than anything else? It's already doing the filter every frame for the whole image, but now it will be doing it once for each texture when it's loaded and natively rendering it at double resolution. I would have thought that would be an improvement on high resolution with the filter applied after, but it would take up 4 times as much memory to store cached textures upscaled 2x.

Have I misunderstood or missed something?

[Exophase]

Sorry, I wasn't clear. I understand why emulators often have software implementations of the GPU, and that running on a CPU is generally less efficient than on the GPU.
What I was wondering was, what is it about it being software rendered that will be more costly than anything else? It's already doing the filter every frame for the whole image, but now it will be doing it once for each texture when it's loaded and natively rendering it at double resolution. I would have thought that would be an improvement on high resolution with the filter applied after, but it would take up 4 times as much memory to store cached textures upscaled 2x.

Have I misunderstood or missed something?

The DS screen is rendered on the CPU, but all scaling and filtering is done by OpenGL ES shaders that are executed on the GPU. The latter happens in parallel with all of the other emulation that takes place on the CPU, and so long as it doesn't take longer than the CPU's workload it doesn't slow things down, at least not beyond a minor impact from increasing main memory contention.

Pre-processing textures would have to be done by the CPU. Applying the filter itself would usually not have a big performance hit because the textures are cached, so long as the game isn't constantly invalidating the textures (which some games do). The performance hit would come in changing the texture's size and representation, which would incur more cache misses in the 3D renderer. The difference is significant enough that it's currently faster to render 8-bit paletted textures than 32-bit direct color textures, despite the former requiring more work per pixel to perform the palette lookup, because it results in textures that are nearly 4 times smaller. Mind you, this performance difference is on Pandora (1GHz Cortex-A8 w/32KB L1 dcache and 256KB L2 cache) which these days is kind of a worst case for memory performance.

2x2 xBRZ textures would have to be double width and height and they'd have to be 32bpp in order to properly capture the gradients. So they'd be nearly 16 times larger than normal textures that can currently be represented as 8bpp, which is a large percentage of textures.

As far as 2D graphics in DS games go, some are rendered as textures by the 3D engine and some are rendered as sprites or layers by the 2D engines. It's feasible to have a pre-processing filter applied to textures. But changing anything about the graphics rendered by the 2D engine would require a completely different sort of renderer, as well as the ability for the emulator to track modifications to 2D graphics. The chances of that being implemented are pretty much zero.

At the very least we're looking to keep screens that don't contain any hires 3D graphics at low resolution (256x192). For a lot of games this will include one of the two screens at any given time. Sometimes both screens.

[nintynuts]

Aha. I suspected the filters might be being applied by the GPU separately, but I hadn't considered that the textures would be going from paletted to full colour. I can see how it will cause major performance issues now, thanks for explaining.

I'm guessing the answer is probably "no" but, have you considered/is it possible to offload this work to the GPU? Maybe it would be possible to 're palette-ise' the upscaled texture with a dithering algorithm and the original palette? Alternatively a variant of xBRZ that sticks to the original palette (might leave banding artifacts, although they would exist in the original)?

Maybe you could load the standard texture as you do now, spin up a thread on the GPU to upscale the texture, then swap out the texture reference once it's converted. Wouldn't that prevent cache misses and prevent slowdown in the CPU? You would see a lower res texture for a couple frames, but it would be just like mipmapping.

Basically, I'm wondering what's the least work you can do to get this working, as I don't expect you want to rewrite the entire rendering engine for GLES.

[Exophase]

I'm guessing the answer is probably "no" but, have you considered/is it possible to offload this work to the GPU? Maybe it would be possible to 're palette-ise' the upscaled texture with a dithering algorithm and the original palette? Alternatively a variant of xBRZ that sticks to the original palette (might leave banding artifacts, although they would exist in the original)?

As I said, the real expense would be in the software renderer having to render the pre-processed textures, generally not in the processing itself. But I can say with little hesitation that any scheme to send the textures to the GPU, process them, and have them read back by the CPU would be slower than doing it on the CPU. GPUs, especially mobile ones, using normal APIs in particular, are not optimized to read back render results to the CPU.

Maybe you could load the standard texture as you do now, spin up a thread on the GPU to upscale the texture, then swap out the texture reference once it's converted. Wouldn't that prevent cache misses and prevent slowdown in the CPU? You would see a lower res texture for a couple frames, but it would be just like mipmapping.

I don't think you understand what I mean by cache misses here.

The 3D renderer accesses textures stored in memory. Those textures are kept in cache as much as possible. Most importantly, the cache helps because of spatial locality of reference. Entire cache lines (64 bytes) are usually loaded at a time which contain multiple texture pixels, and there's a good chance that more of them will be used in the future.

When you make textures bigger they need more accesses out to main memory to be loaded into the processor's cache and this hurts performance.

Basically, I'm wondering what's the least work you can do to get this working, as I don't expect you want to rewrite the entire rendering engine for GLES.

Meanwhile I'm wondering what's the most amount of people who want this feature and can handle the performance degradation - if it's just you that's kind of a hard sell While I do think that the performance of the texture filtering wouldn't necessarily be critical, I'm also not sure I could just adapt some slow existing routine for it instead of writing an optimized NEON version. And I don't even yet know the algorithm in the first place.

Any mention of rendering any DS stuff on OpenGL ES is a big can of worms for the time being.