When you get Moiré from the sensor of a camera or scanner, then there is nothing much you can do.
But extreme Moiré effect from resizing is not really Moiré. It's aliasing. And it typically happens because the resizing happens in a non-linear colorspace and/or with a poor kernel. Unfortunately this mistake is quite common in many image processing software.
Technically it's not, but your description is, because of the words "grid of dots", which means there's white space between the dots (as dots are circular), a necessary requirement for the Moiré effect. We see this in image sensors as well, where all photosites have a tiny amount of space between them.
Image resizing isn't a grid of dots, though. At the data level, pixels don't have space between each pixel to cause Moiré effects. Instead, interpolation of the colors leads to patterns similar to, but distinct from, the Moiré effect.
The Moiré effect is "not really there" in that it's a perceptual effect: you can reproduce it with an SVG with the same dot pattern on two layers, and transforming one of them a little. A human observer will "see" patterns that aren't actually there, and as you zoom in the effect disappears (if you're now thinking "because it's on a monitor!", then no: it's not. The same is true for two physical transparencies with the same dot pattern placed slightly shifted on an old school projector, and then changing the zoom on the projector)
Antialiasing is a data property: it can result from lossy interpolating of pixels to a new grid, and is there no matter how much you zoom in because zooming in on a pixel doesn't change its color.
The confusion comes from talking about Moiré patterns on grid based displays. Those will "lock in" or even intensify the effect, but they don't cause the effect.
There's not much you can do to fix it in software, but you can fix it in the optics. You can add a Moiré filter to the camera, which is an optical low-pass filter, ensuring that the image that reaches the sensor does not have any spatial-frequency components above the Nyquist frequency of the sensor (ie, twice the pixel pitch).
I obtained a similar effect with The Gimp and the wavelet decomposition plugin[1] that splits the picture in various layers containing details at different granularity. You can then simply remove the "bad" ones. On portraits, you can remove freckles and wrinkles very easily this way, too.
Done since the mid-1960's :) but, nevertheless, always an interesting read!
This is classical (as in pre-GPU AI/ML) Image Processing. The reference text book in the field, "Digital Image Processing" by Gonzalez et al, shows how to do it: like the pioneers at JPL's Image Processing Laboratory (est. 1965) denoised Mars pictures from the Mariner probes using Frequency Domain methods like the posted article uses - see an example below.
It's so classical, it can be done without a digital computer at all, using fourier optics! The true fastest fourier transform, done at the speed of light.
At the focal plane of a lens, an image is converted into its spatial fourier components. By adding blocking elements at that focal plane (eg. an aperture, or a piece of glass with marker dots), unwanted frequency components can be cut out from the image.
I wonder what would be the more systematic approach than manually painting over the fft image? That constellation of peaks in fft should be reasonably easy to recognize (semi-)automatically, then you'd need to figure out good mask for those.. is it a circular blob, or maybe diamond/star shape? And what size is best? Etc etc.
It has been a long time since I have done anything in this domain (20+ years), but if I understand correctly, the original article author is essentially filtering out high frequency signals by painting over the bright FFT spots, essentially creating a localized low-pass filter or maybe band-pass filter.
FFT essentially just transforms data into its frequency domain, so I assume the representation shown in the article is just that, brighter spots indicate higher frequency information (which would come from all the little spaced dots), so killing those essentially smooths the noise out of the data.
I imagine that the same effect could be automatically achieved by identifying which frequencies to filter out to get the desired effect.
I dont believe that by painting over the bright FFT spots you are necessarily removing the high frequencies. Instead, you are removing the strong frequencies from the image. The strong frequencies cause peaks in the FFT image.
Its the repetitive nature of the moire effect that cause peaks in the FFT image.
Indeed, the frequencies get higher further away from the center of the FFT image with the center being the "DC" component of the signal. The brightness, as you note, correspond to the amplitude of a given frequency.
you can think of it as a polar plot where:
radius = frequency,
angle = angle (as in points along the horizontal direction in the FFT correspond to e.g. vertical stripes in the image),
brightness = power
There is an imaginary part to the FFT that contains the phase-shifts needed to add everything back up into the original image. You may have noticed that the "FFT of the image" is symmetric, so something else should be needed to compute the inverse.
Ahhh, so perhaps the position indicates the frequency and the brightness indicates the magnitude of the frequency? That makes sense - in that case the atricle's approach feels more like you said, a manually applied band pass or comb filter, i.e. it either removes or attenuates signals within certain frequency ranges.
Another nice property of the FFT is that convolution in image space is multiplication in frequency space, so multiplying the FFT with an image of a Gaussian (or any other linear kernel such as an edge detector) is exactly equivalent to applying the filter on the original image.
Theoretically, you're attempting to remove the artifacts that were added back in by the halftone printing process and recover the original image. If you knew the exact noise spectrum that the halftone print added into the image, you could invert it. For example with some scans of halftone prints of known images, you could reconstruct that noise spectrum and remove it as best as possible. This would preserve, to your best ability, whatever high-frequency data existed in the original image.
If we were sure that the halftone resolution was smaller than the relevant details of the photo, then we could just use a low-pass filter (a circular aperture), removing all the high-frequency components from the halftone print, but keeping the details of the photo. However in this case, the photo has details that are at the same scale as the halftone.
So the next best thing is to try to infer the halftone noise spectrum and subtract it, which is hard because it's mixed in with the photo, and we don't know which spectral peaks are from the photo and which are from the halftone process. What we can guess is that the legitimate high-frequency information in the photo should be pretty random, meaning it will be smeared out across the fourier spectrum, while the halftone print will be strongly periodic - concentrated into dots that are nearly symmetrically distributed around the origin (with some radial pattern that depends on the print process), away from the low-frequency center point.
Based on this we guess that any big dots that are symmetrically placed around the origin are probably more halftone noise than image signal, and any grey areas between them are probably more real image data than noise. Then we fit a noise filter to the dots with a regression, doing our best guess to invert the noise, bringing the whole high-frequency zone to approximately a uniform grey.
And then we hope that the picture isn't a photo of a herd of small spotted leopards.
> All the demoireing guides tell you to apply a sharpening filter after this to compensate, but that’s like reheating cold pizza. You will never get back to where you were before, no matter how hard you try
Actually the noise from the moire can fake detail that wasn't there in the first place. Put film grain or a raster on top of a blurry image and it will subjectively improve in quality.
"This is why nobody will allow you to wear stripes on television:"
This used to be cardinal rule in television broadcasting but it's much less a problem nowadays with the increased resolution/line rate of HD TV and video processing (filters) designed to eliminate it.
Nevertheless, whilst once discouraged, moiré was often a useful tool as camera and CCU (Camera Control Unit) operators would use it to focus an image—the more pronounced the moiré the sharper the camera's focus.
As bad as moiré is perceived a much greater cardinal 'sin' in both television broadcasting and photography is lateral inversion. This is where the image has been swapped in the horizontal direction (in some places it was a dismissible offence), as it's a complete distortion/misrepresentation of the image, which in some instances, may go undiscovered—there being no visual clues to indicate the problem.
Lateral inversion is very obvious when writing, street signs etc., appear backwards but for some reason it's very much less so when images of humans are involved. Despite the fact that laterally-inverted images put men's clothes on women and vice versa—as blouses, shitrts, coats and pants flies appear the wrong way around—few people seem to notice.
There's much evidence for this, one I often cite is that there are images from WWII on the US National Archive by the US Army Signal Corps that are still laterally inverted after 70-plus years that no one has bothered to correct (it's not a recent scanning error either as the Signal Corps logo (which is embedded within the photo) is not laterally inverted but the image content is).
BTW, laterally inverting a film image reduces the resolution as the negative or slide is no longer in the normal focus plane.
Here's my favorite "alternative" explanation of moiré:
sin(a) * sin(b) = 0.5 * (cos(a - b) - cos(a + b))
You may remember this formula from high school. The arguments a and b are basically frequencies. When you multiply two frequencies a and b, you will get their difference cos(a - b) and their sum cos(a + b). If two frequencies are similar (e.g. 101Hz and 100Hz) you get one very low frequency (101Hz - 100Hz = 1Hz) and the other one will be much higher number (101Hz + 100Hz = 201Hz). It's hard to see that higher frequency 201Hz, you have to come really close, but it is easy to see that low frequency, that 1Hz would manifest as 1 dark blob across entire area.
Hang on then, if you're just notching out certain frequencies by painting over them in a 2D FFT could you do the same thing by treating the image like it was raster-scanned video and passing it through a notch filter at the Moiré frequency, or for that matter a comb filter? After all that's how you remove PAL subcarrier speckle from composite video.
A certain YouTube creator wears striped shirts on camera a lot, but I feel like he should probably know better. I wonder if it’s an intentional, subtle troll?
A sage reminder to all of us that sometimes it's better to upvote & listen (or in this case heart/retweet) to see if a better response surfaces than to offer our two cents.
But extreme Moiré effect from resizing is not really Moiré. It's aliasing. And it typically happens because the resizing happens in a non-linear colorspace and/or with a poor kernel. Unfortunately this mistake is quite common in many image processing software.