scale2x / scale3x shaders

[Stephane Hockenhull]

Using vec3 instead of vec4, the scale2x code reach 125-130ms. Not usable (far from it) but interesting

Spoiler

current Fragment shader :

precision mediump float;

varying vec2 v_texCoord[5];

varying vec2 pos;

uniform sampler2D s_texture0;

uniform vec4 u_param;

void main()

{

		vec3 E = texture2D(s_texture0, v_texCoord[0]).xyz;

		vec3 D = texture2D(s_texture0, v_texCoord[1]).xyz;

		vec3 F = texture2D(s_texture0, v_texCoord[2]).xyz;

		vec3 H = texture2D(s_texture0, v_texCoord[3]).xyz;

		vec3 B = texture2D(s_texture0, v_texCoord[4]).xyz;

		vec2 p = fract(pos);

		vec3 tmp1 = p.x < 0.5 ? D : F;

		vec3 tmp2 = p.y < 0.5 ? H : B;

		vec3 tmp3 = D == F || H == B ? E : tmp1;

		gl_FragColor.xyz = tmp1 == tmp2 ? tmp3 : E;

}

have you tried with

lowp​ vec3 E = ....

?



also, try moving p=fract(pos); to the top away from its first use



vec3 E = ...

could be at the bottom of the texture lookups to put more stuff between D,F,H,B first use



this may reduce data dependency stalls if the shader compiler doesn't reorder thing properly.

Spoiler

precision mediump float;

varying vec2 v_texCoord[5];

varying vec2 pos;

uniform sampler2D s_texture0;

uniform vec4 u_param;

void main()

{

		lowp​ vec2 p = fract(pos); // away from its first use

		lowp vec3 D = texture2D(s_texture0, v_texCoord[1]).xyz;

		lowp​ vec3 F = texture2D(s_texture0, v_texCoord[2]).xyz;

		lowp​ vec3 H = texture2D(s_texture0, v_texCoord[3]).xyz;

		lowp​ vec3 B = texture2D(s_texture0, v_texCoord[4]).xyz;

		lowp​ vec3 E = texture2D(s_texture0, v_texCoord[0]).xyz;  // put H,B further away

		lowp​ vec3 tmp1 = p.x < lowp​ 0.5 ? D : F;

		lowp​ vec3 tmp2 = p.y < lowp​ 0.5 ? H : B;

		lowp​ vec3 tmp3 = D == F || H == B ? E : tmp1;

		gl_FragColor.xyz = tmp1 == tmp2 ? tmp3 : E;

}

 

[Stephane Hockenhull]

95ms

Spoiler

precision mediump float;

varying vec2 v_texCoord[5];

varying vec2 pos;

uniform lowp sampler2D s_texture0;

uniform vec4 u_param;


void main()

{

lowp vec3 E = texture2D(s_texture0, v_texCoord[0]).xyz;

lowp vec3 D = texture2D(s_texture0, v_texCoord[1]).xyz;

lowp vec3 F = texture2D(s_texture0, v_texCoord[2]).xyz;

lowp vec3 H = texture2D(s_texture0, v_texCoord[3]).xyz;

lowp vec3 B = texture2D(s_texture0, v_texCoord[4]).xyz;

lowp vec2 p = fract(pos);


lowp vec3 tmp1 = p.x < 0.5 ? D : F;

lowp vec3 tmp2 = p.y < 0.5 ? H : B;

lowp vec3 tmp3 = D == F || H == B ? E : tmp1;

gl_FragColor.xyz = tmp1 == tmp2 ? tmp3 : E;

}

 

[Stephane Hockenhull]

80ms

Spoiler

B) == vec3(0.0)) ? E : tmp1;

}

precision mediump float;

varying vec2 v_texCoord[5];

varying vec2 pos;

uniform lowp sampler2D s_texture0;

uniform vec4 u_param;


void main()

{

	lowp vec2 p = fract(pos);

	lowp vec3 E = texture2D(s_texture0, v_texCoord[0]).xyz;

	lowp vec3 D = texture2D(s_texture0, v_texCoord[1]).xyz;

	lowp vec3 F = texture2D(s_texture0, v_texCoord[2]).xyz;

	lowp vec3 H = texture2D(s_texture0, v_texCoord[3]).xyz;

	lowp vec3 B = texture2D(s_texture0, v_texCoord[4]).xyz;


	lowp vec3 tmp1 = p.x < 0.5 ? D : F;

	lowp vec3 tmp2 = p.y < 0.5 ? H : B;

	gl_FragColor.xyz = ((tmp1 - tmp2) != vec3(0.0)) || ((D - F) * (H -

h34r:

 

[Stephane Hockenhull]

45ms (GB, SNES, 512x240)


62ms (GBA or fullscreen)


should be playable

precision mediump float;

varying vec2 v_texCoord[5];

varying vec2 pos;

uniform lowp sampler2D s_texture0;

uniform vec4 u_param;


void main()

{

	lowp vec3 E = texture2D(s_texture0, v_texCoord[0]).xyz;

	lowp vec3 D = texture2D(s_texture0, v_texCoord[1]).xyz;

	lowp vec3 F = texture2D(s_texture0, v_texCoord[2]).xyz;

	lowp vec3 H = texture2D(s_texture0, v_texCoord[3]).xyz;

	lowp vec3 B = texture2D(s_texture0, v_texCoord[4]).xyz;


	if ((D - F) * (H -  == vec3(0.0)) {

		gl_FragColor.xyz = E;

	} else {

		lowp vec2 p = fract(pos);

		lowp vec3 tmp1 = p.x < 0.5 ? D : F;

		lowp vec3 tmp2 = p.y < 0.5 ? H : B;

		gl_FragColor.xyz = ((tmp1 - tmp2) != vec3(0.0)) ? E : tmp1;

	}

}

[/CODE]


h34r: h34r: h34r: h34r: h34r:

 

[gunrock]

Niiiiiiiiiiiiiiiiiice. I love to optimisation making steady iterative strides like this!


Well done, dude.

 

[Stephane Hockenhull]

out of curiosity I also tried to eliminate the:


lowp vec3 tmp1 = p.x < 0.5 ? D : F;


lowp vec3 tmp2 = p.y < 0.5 ? H : B;


lines by swapping the texture coordinates in the pixel shaders:


[...]


lowp vec3 D = texture2D(s_texture0, v_texCoord[1] + s.xz).xyz;


lowp vec3 F = texture2D(s_texture0, v_texCoord[2] - s.xz).xyz;


[...]


and as it has been documented in the PVR optimizing docs, its a horrible idea to programmatically change the texture coordinates in the pixel shaders.


45ms -> 130ms


dependent texture reads are a thing to really avoid on SGX ( modifying texture coordinates = BAAAAAAAAAD ).


in other words, forget bumpy water "reflexions" waving done in the pixel shader, you should wave the whole pre-sub-divided water plane geometry from the vertex shader instead.

 

[Stephane Hockenhull]

In C, ?: is called the "conditional operator".. since it's the only ternary operator (three arguments) it's often just called "the ternary operator."


(...)


Personally I never use the operator in my C code, I guess I just can't stand the appearance of it.. but nothing is really stopping me or anyone else from making a macro to hide it.

I think a macro will obscure it more by making it look like a function: all function parameters get evaluated, a macro will NOT evaluate the unselected value. this will cause some confusion.


but I agree its generally best to avoid using more than 1 in a line of code, its get messy pretty quickly.


as an example of what not to do, the world generator in BBQ has this little gem of horror:

		uint tile =

			gp[1] > ground_level ?

			tile_air() :

				((ground_level > 0) ?

					(

						(gp[1] < (ground_level-2) ? tile_stone() :

							(ground_level < 40) ? (

								(gp[1] == ground_level ? tile_grass() : tile_dirt() )

							) : (

								(ground_level < 52) ? tile_mountain() : tile_snow()

							)

						)

					): (

						(gp[1] < (ground_level-2) ? tile_stone() :

							tile_sand()

						)

					)

				);

*actual code

 

[Eric Jardim]

Hi, I kind read this post a bit latelly, but I think I can help.


I've been working with high quality GPU scaling filters this summer (in Brazil the summer is Jan-Feb).


I still do not understand exactly what are you doing with this scalers, but, as far as I know, there is no need to do "if" to perform bilinear filtering (scaleNx). But, again, I still need to learn more what you are doing.


Thanks,

 

[sebt3]

http://scale2x.sourceforge.net/algorithm.html


if it was for bilinear filtering the hardware scaler would do

 

[skeezix]

I really like this experiment though; I keep thjinking I'd like to take a stab at it, but never get around to it, but I'm glad you did.. definately a starting point, and woudl be the way I'd have started too Really fun little project, and coudl be really useful once solved.


jeff

 

[Eric Jardim]

if it was for bilinear filtering the hardware scaler would do

Ops! Sorry !


Anyway, I think I can still help. I saw the code have a lot of "ifs", and it is probably slowing the result. I think the natural speed up is done replacing the conditionals for texture acesses and masking. Think this p.x, p.y, A, B, etc as texture coordinates and precompute all the situations so there is no need to perform the "if". Then mask the result (e.g. E = v1 * D + v2 * F) where v1 and v2 are 0 or 1 depending what was read in texture.


Other thing: I think the texture offsets computed in the vertex shader can be done once per render pass (not every pixel), again passed as texture. Have you tried this? I would try it first.


BTW, I also know some tricks to implement the cubic Catmul-Rom filter (Keys) with 9 texture queries and the bicubic spline with only 4 texture queries. There are very nice and cheap filters. There are better (expensiver) ones. We can see what is possible.


So.... how can I test this shaders? Is there any quick testing kit ?

 

[Stephane Hockenhull]

Think this p.x, p.y, A, B, etc as texture coordinates and precompute all the situations so there is no need to perform the "if".

dependent computed texture coordinate accesses (ones with code before them) are slower than conditional on SGX


using textures as lookup tables is really slow.


the source image is regenerated every frame, so would the precompute lookup, that would eliminate any and all savings in using the GPU.

Other thing: I think the texture offsets computed in the vertex shader can be done once per render pass (not every pixel), again passed as texture. Have you tried this? I would try it first.

the texture offsets computed in the vertex shader ARE computed once per render pass, that's why they're in the vertex shader, not the pixel shader.


this renders a SINGLE 2D QUAD (two triangles) per render pass.


and again, texture accesses are slow, you need to minimize them.

BTW, I also know some tricks to implement the cubic Catmul-Rom filter (Keys) with 9 texture queries and the bicubic spline with only 4 texture queries. There are very nice and cheap filters. There are better (expensiver) ones. We can see what is possible.

texture queries are what slows down the code the most, going to 9 texture queries with a Catmull-Rom would be atrociously slower.


the goal of scale2x and similar is not to provide interpolation of real images but to reconstruct curves in ink-lines as humanly percieved in low-resolution artificial cartoony image sources such as 16-bit era 2D games when upscaling them for display at larger resolutions.


we don't want interpolation, we want detail reconstruction like this:


http://en.wikipedia.org/wiki/Hqx