Digital BW, The Print

On Dec 6, 2004, at 7:04 PM, Anthony G. Atkielski wrote:

>
> Roger Howard writes:
>
>> No - assuming no loss, there are real limits to how far compression 
>> can
>> take you. It's closely related to Shannon's Law.
>
> Yes, but we aren't hitting those limits, because practical compression
> algorithms can't do optimal compression.
>
> For example, many modern algorithms do Walsh-Fourier transformations to
> compress data.  For practical reasons, these do these only on tiny
> chunks of image data at a time; but it's possible to do a single
> transformation on the _entire image_.  This latter transformation
> results in much better compression ... but it takes an extremely long
> time to complete.  And so nobody actually does it.
>
> Exactly how much of a gain this represents, I don't know, because 
> nobody
> actually does it.

Again, there are limits, and plenty of people have tested them (not in 
a typical design production environment of course)... there are 
seriously diminishing returns until you start getting into other 
concepts that aren't compression - like using huge codebooks and clever 
indexing to transmit large sequences of arbitrary data as just an 
offset to lookup in your codebook (the various compression forums see 
claims based on these techniques every year or so).

Typically, lossless compressors bottom out within maybe 10% of each 
other in compression efficiency... so there's not much to be gained 
even by huge increases in the resources you throw at it. Look beyond 
image compression at the general data compression market - folks like 
Aladdin and Pkware have been struggling to eke out minor improvements 
in efficiency without huge losses in performance for years...

The real gains are in lossy compressors, but those have to be treated 
with care.

>> Today, the best compressors for really great efficiency on really high
>> rez materials tends to be wavelets... JPEG2000 and MrSid are both
>> popular implementations. The artifacts are much less obnoxious than
>> DCT, and the compression ratios are much better. For really large
>> images, unless you regularly view them at 100% then a wavelet
>> compressor (or even DCT) can be used pretty heavily without losing too
>> much significant detail.
>
> If you keep the frequencies high enough, you should be able to get
> effectively lossless compression.  The frequencies would have to be
> infinitely high to get perfect lossless compression, though (if I'm not
> mistaken).

Oh for sure - this is the same of course for most lossy compressors... 
but it's still critical to consider them lossy, unless they are 
provably lossless in a very literal way.

> In any case, as I've said, I think the future is in more bandwidth and
> more storage space, not greater compression.  As you point out 
> yourself,
> Shannon proved that one can only do so much with (lossless--meaning
> perfectly reversible) compression.

I totally agree, which is why I've steadfastly been doubtful about the 
potential successes for JPEG2000 - I just don't see the demand today, 
as nifty as it might all seem. I have a 7 year old image server based 
on FlashPix (yuck!) that can do thumbnails dynamically from a 80MB 
master. Disk is cheap. And there are no really hot JPEG2000 systems in 
wide deployment - lots of great demos and whitepapers... but it would 
have to be supported as a new media type - rather than just a better 
compressed static image - to really benefit in most applications.

Ok now we're way OT I'm sure :)

-R