Plug-in SDK

Upcoming mother of all image editors
andydansby
Posts: 161
Joined: Fri Oct 29, 2010 6:00 am
Location: Syracuse, NY

Re: Plug-in SDK

Post by andydansby »

I think I have worked out the bug in my edge preserving smoothing filter. A simple and fast, however the output similar to a bilateral, however the logic is a range reducer over small jumps. It is linear edge preserver and I'm not sure if it's been published before (if it has I have not found it). I will release it soon as I am finished with tweaking it.

Here's a teaser
original
original
clawn.jpg (53.23 KiB) Viewed 4363 times
original
enhanced image
enhanced image
clawn_Final.jpg (118.61 KiB) Viewed 4363 times
enhanced

Andy Dansby

Fluffbutt
Posts: 1060
Joined: Sun Apr 24, 2011 3:05 am
Location: No longer here

Re: Plug-in SDK

Post by Fluffbutt »

Interesting..

Can you try something - do it at "higher" settings - it looks like it might turn a photo into a cartoon cell drawing (cells used in animation)..

Nice edge preservation and good smoothing.

andydansby
Posts: 161
Joined: Fri Oct 29, 2010 6:00 am
Location: Syracuse, NY

Re: Plug-in SDK

Post by andydansby »

Here an image with 10 radius and 50 Lambda (which is 100 in euclidian distance +50 and -50) color distance from center pixel. The way the algorithm works is using the euclidian distance, which is simply the center point + - whatever is set in lambda. So what will happen is the further the color distance, the more chance it will just blur.
clawn_Final 10 radius 50 lambda.jpg
clawn_Final 10 radius 50 lambda.jpg (98.87 KiB) Viewed 4339 times
I have been reading up on some cartooning effects in the last few weeks or so (it's been on my mind) and it seems like a decent edge preserver is the key to a base layer. It might not be that far of a stretch getting this to work like a cartooner. This filter seems to make a good sharpening enhancement.
original
original
rocky.jpg (55.68 KiB) Viewed 4339 times
original
enhanced sharpen
enhanced sharpen
rocky_Final.jpg (181.4 KiB) Viewed 4339 times
sharpen enhanced


Another immediate effect is that within certain constraints, it effectively removes mosquito noise in jpeg cartoon images.

You may have to zoom in to see the mosquito noise
original
original
Smiley_face_example_true_C_.jpg (4.79 KiB) Viewed 4339 times
original

oops, I can't add more than 4 images, see the next post

Andy Dansby

andydansby
Posts: 161
Joined: Fri Oct 29, 2010 6:00 am
Location: Syracuse, NY

Re: Plug-in SDK

Post by andydansby »

continued
enhanced
enhanced
Smiley_face.png (18.67 KiB) Viewed 4340 times
enhanced
without mosquito noise

Andy Dansby

Fluffbutt
Posts: 1060
Joined: Sun Apr 24, 2011 3:05 am
Location: No longer here

Re: Plug-in SDK

Post by Fluffbutt »

andydansby wrote:Here an image with 10 radius and 50 Lambda (which is 100 in euclidian distance +50 and -50) color distance from center pixel. The way the algorithm works is using the euclidian distance, which is simply the center point + - whatever is set in lambda. So what will happen is the further the color distance, the more chance it will just blur.
Curious - how do you find/define the centre pixel to count from? Do you scan for similar colour - is there a control for how "fuzzy" that "similar colour" discovery acts?

I LIKE that clown - he does look like a 'toon,.

I understand your problem or point about the Lambda distance - you don't so much preserve the edges as just no go over them to need preserving.

You could add a check for deviation from the "scan for similar colour to find centre pixel" algorithm - so that, for e.g., if it's scanned for yellow and found the centre x/y value and your 50 Lambda works in one direction but the other would put it over a dramatic colour change then it ignores the λ and stops at the major colour change border.

I don't know If I worded that properly.
I have been reading up on some cartooning effects in the last few weeks or so (it's been on my mind) and it seems like a decent edge preserver is the key to a base layer. It might not be that far of a stretch getting this to work like a cartooner.
This sort of gets back to the:

sample colour of X/Y centre -> store
sample colours around centre // How? How do you define where to search"
compare to allowed deviation from stored value // "allowed deviation" = a "fuzziness value we define"
IF within deviation THEN make current pixel value = stored value ELSE cancel current search
REPEAT until current defined area is completed // based on knowing the centre coordinates


That would make a yellow are be ALL the same colour yellow but would stop at a colour change defined by us.
Essentially, it's just a user controllable and configurable "reduce colours" thing?

Another method might be to find edges by defined differences in colour, then just make a one pixel (or defined value) black edge exactly where the colour changes by more than the fuzzy amount.
I mean.. AFTER setting the colours to be the same, or as you've done.. if you added the black line pixel-edge to your clown at the defined borders of your colour areas, it would increase the 'toon look.

Another immediate effect is that within certain constraints, it effectively removes mosquito noise in jpeg cartoon images.

You may have to zoom in to see the mosquito noise

Andy Dansby
I noticed that - very useful.
Also good for spots of noise and dust, and those little white dots that some DAP styles leave behind!

andydansby
Posts: 161
Joined: Fri Oct 29, 2010 6:00 am
Location: Syracuse, NY

Re: Plug-in SDK

Post by andydansby »

Hi Fluff:

Curious - how do you find/define the centre pixel to count from?
I just grab the center pixel of the kernel, since this algorithm is separable, within the xy loop, I just have an i loop that goes over the kernel and of course halfway between the start and end if the i loop is the center pixel. I grab its value in a separate variable and sum the rest of the values unless it exceeds the value of the center pixel by the control adjusted in lambda. It's probably easier to show the source code than to explain it, soon after I release the plugin, the source code will go into my intermediate image processing guide in the edge preserving section.
Do you scan for similar colour - is there a control for how "fuzzy" that "similar colour" discovery acts?
Right now, the algorithm runs in the RGB colorspace, which is a poor way to do things for similar color, however RGB is fast to drum up code and then I can apply to other color spaces. I plan on working in the LAB colorspace, but also may try the YCbCr (full bandwidth) colorspace as well. LAB space works great for color similarity, RGB not so much.


I LIKE that clown - he does look like a 'toon,.
I'm pretty sure it's an image found in Matlab - like I can afford that, but grabbed it off the internet, i like the looks of the image.
I understand your problem or point about the Lambda distance - you don't so much preserve the edges as just no go over them to need preserving.
You are dead on correct with that analysis, it skips the edges by not calculating them. It's far easier than I thought it was, a real simple code. My useage term for Lambda may not be entirely accurate, as I just kept the term from something else I was working on. It's just a measurement on how far the distance gradient from 0 to 1 to continue to add other pixels (in RGB), you might note that I have changed the scale from 0-100 to 0-1 now, as I am trying to future proof if Oscar wants to go to 16 bit image support.
You could add a check for deviation from the "scan for similar colour to find centre pixel" algorithm - so that, for e.g., if it's scanned for yellow and found the centre x/y value and your 50 Lambda works in one direction but the other would put it over a dramatic colour change then it ignores the λ and stops at the major colour change border. I don't know If I worded that properly.
I might have to think about that one, so we are taking about going in a + Lambda or - Lambda direction only, a one way filter. That would be an interesting variation.

As for the cartoon effect I saw a matlab solution, I'll have to look at the logic for it, but first I must play around in the LAB space.

Andy Dansby

Fluffbutt
Posts: 1060
Joined: Sun Apr 24, 2011 3:05 am
Location: No longer here

Re: Plug-in SDK

Post by Fluffbutt »

A very interesting read, thanks!

It occurred to me that a 'toon filter would be very similar to a self-restricting JPG conversion.

JPG works by scanning blocks of similar colours and "making them all the average of the colours" with the range of the macroblock. The JPG quality helps determine how big the sample block is and how far our of average it can include and "make average".

That, coupled with an initial edge detection, which could either be applied over the finished output OR used to self-restrict how far the blocks go, would make a pretty interesting 'toon-ish filter.

Like this, if it works:

◙◙◙◙◙◙◙◙◙◙/☼☼☼☼☼☼/
◙◙◙◙◙◙◙◙◙/☼☼☼☼☼☼/
◙◙◙◙◙◙◙◙/☼☼☼☼☼☼/
◙◙◙◙◙◙◙/☼☼☼☼☼☼☼/
◙◙◙◙◙◙/☼☼☼☼☼☼☼☼/
◙◙◙◙◙/☼☼☼☼☼☼☼☼☼☼/
◙◙◙◙/☼☼☼☼☼☼☼☼☼☼/
◙◙◙/☼☼☼☼☼☼☼☼☼/

◙ Colour blocks by similarity, average colour applied after filter ends, with a control to define how far off average is still included.
"/" = a detected edge, either only used to end the colour search early OR to make a 1/2/3 pixel wide line between colour groups
☼ next detected "similar colour", after the edge between colours.

In this case here, I don't mean "edge detection" as edges of walls and cars, trees, eyes, etc - I mean a change in colour outside of the range considered as "averageable"
So and edge might actually occur between sky blur and mid or dark blur, depending on where the control was for that factor.

Above, the blocks could be dark blue, the "/" a 2 pixel line taking 1 pixel from each colour, the sun is light blue.

That's pretty much what the early cell cartoons looked like - it might mean some trial an error for the user, to find the exact control value for best 'toonerisation.



Kind of what segmentation does, on a wider scale, but with more control over what and where (seg seems a little to random for cartooning)
So you could make a whole tree turn green with a fine black edge or have the tree show as 2 or 3 green shades, each with fine black edges.

Edit - I've edited this about 7 times now for damn typos - if you see any more, just ignore them.
My mind works in ODD ways - I was typing "omnivore" in something else, looked up - I had actually typed "pornovore" ??!!!

JEL
Posts: 295
Joined: Fri Jan 06, 2012 9:35 pm
Contact:

Re: Plug-in SDK

Post by JEL »

andydansby wrote:I haven't even tried it with greyscale images, didn't even cross my mind. Once I post the source code, you will see that the routine is based entirely on standard deviation and mean of the source image and applying to the target image.
Would it be possible to make a different version of your plugin that worked on contrast/brightness only? IE one where you could take your desired histogram and copy it to other images, such that highs, mids and lows where modified to look like your histogram-source (kind of like a variation of the histogram-equalize block. One where you had more control over how the histogram was spaced out)

I tried via an RGB-split flow, but that didn't seem to do much.
(I set up 3 flows with your plugin in each color-channel, but the changes I could achieve was very minimal. Your plugin is very effective on color though)
DAP (AOPs): http://jelstudio.dk/DAP/
PhotoReactor (flows, effects and scripts): http://jelstudio.dk/PhotoReactor/

andydansby
Posts: 161
Joined: Fri Oct 29, 2010 6:00 am
Location: Syracuse, NY

Re: Plug-in SDK

Post by andydansby »

Sorry I have replied sooner, the winter here is upstate New York is getting to me, I just want to hibernate.

Fluffbutt, when I read Pornovore, I laughed so hard, the people that were near me physically backed away. Perhaps because they thought I was crazy or something. I'm going to have to drop that in some conversation something just to see if someone's listening.

I will after I get caught up on some other things, will seriously consider trying out a filter like that using edge preserving to create a cartoon filter. I've wanted to create one since I started to play around with the oilify filter and other non-photo-realistic image processing. I would like to update my guides (beginner and intermediate image processing), play around and finally release my latest filter, release my code for not only my latest filter, but the color transfer filter. Of course this doesn't go as fast as it sounds, I have work, family and house repairs etc.

As for edge detection, I think what you described is edge preservation. Edge detection is something akin to sobel, prewitt etc, which would be interesting to combine that with a algorithm for cartooning. Edge preserving is more like Bilateral, Symmetric Nearest Neighbor, my latest filter etc which uses color differences to determine an edge and smooth anything that is not considered an edge.

I tried to make my latest filter from an RGB filter to YCbCr and I did get different results, but nothing more than a very slight difference. I also tried using LAB and for some reason, killed the blue color range and it shouldn't. A little frustrating. I might try the LAB space described in the Reinhart paper.


Hi Jel:
Would it be possible to make a different version of your plugin that worked on contrast/brightness only? IE one where you could take your desired histogram and copy it to other images, such that highs, mids and lows where modified to look like your histogram-source (kind of like a variation of the histogram-equalize block. One where you had more control over how the histogram was spaced out).
I certainly can make the filter copy luminance from one image to another. That should be fairly easy to do within the same algorithm, which is just mean and standard deviation of image B applied to image A. I'm not sure how to copy contrast from one image to another, I don't know how contrast can be measured, but I sure that someone has figured it out somewhere.

As for making the filter work for a histogram-equalize block, I haven't figured out histogram-equalize as of yet, though as of lately I've been working more on edge preserving smooths.

Talking about kernel driven filters, AHE and it's derivative CLAHE have been driving me insane for ages. I have not been able to get one to look right and this is after studying others code and algorithms. I would love to get that to work and create a grunge type effect.

Andy Dansby

andydansby
Posts: 161
Joined: Fri Oct 29, 2010 6:00 am
Location: Syracuse, NY

Re: Plug-in SDK

Post by andydansby »

I'm releasing my latest filter as a beta test. It is in beta not because of any problems with the code, but the algorithm sometimes is not as edge preserving as I want. It is what it is. I am still playing around with color spaces on this filter, but experiments with CIE LAB killed colors (blue) and YCbCr seemed to have only the smallest of differences. Reinhart LAB make the colors funky. On the plus side, while the filter works by separating in RGB fashion, it seems to be zippy quick.

The filter name is cruddy, but descriptive on how it works. Spacial Range Reduction. Within the radius, it will reduce the range of colors that fall within the Lambda control. The filter is linear and separable, performing Horizontal and then Vertical smoothing steps. A full kernel (x & y) may be another experiment I may perform as well as troubleshooting some of the other color spaces. Future releases may include a strong algorithm, full kernel and other color spaces, however I did want to place the filter out there for others to play with.
SRR 32.zip
32 bit version
(47.43 KiB) Downloaded 261 times
32 bit version
SRR 64.zip
64 bit version
(52.04 KiB) Downloaded 310 times
64 bit version


Enjoy
Andy Dansby

Post Reply