Data compression

Data compression, a fundamental topic of computer science, is the process of encoding data so that it takes less storage space or less transmission time than it would if it were not compressed. This is possible because most real-world data is very redundant or not most concisely represented in its obvious form.

One very simple means of compression, for example, is run-length encoding, wherein large runs of consecutive identical data values are replaced by a simple code with the data value and length of the run. This is an example of lossless data compression, where the data is compressed in such a way that it can be recovered exactly. For symbolic data such as spreadsheets, text, executable programs, etc., losslessness is essential because changing even a single bit cannot be tolerated (except in some limited cases).

In other kinds of data such as sounds and pictures, a small loss of quality can be tolerated without losing the essential nature of the data, so lossy data compression methods can be used. These frequently offer a range of compression efficiencies, where the user can choose whether he wants highly-compressed data with noticeable loss of quality or higher-quality data with less compression. In particular, compression of images and sounds can take advantage of limitations of the human sensory system to compress data in ways that are lossy, but nearly indistinguishable from the original.

Many data compression systems are best viewed with a four-stage model.

Closely allied with data compression are the fields of coding theory and cryptography. Theoretical background is provided by information theory and algorithmic information theory. When compressing information in the form of signals we often use digital signal processing methods. The idea of data compression is deeply connected with statistical inference and particularly with the maximum likelihood principle.

Data compression topics:

Common Data compression algorithms:

Lossless Compression
- Run-length encoding
  - PackBits
  - PCX
- Minimum-redundancy coding, also called entropy encoding
  - Huffman coding (simple entropy coding)
  - Arithmetic coding (more advanced entropy coding; encumbered by patents as of October 2001)
- Dictionary coders
- Burrows-Wheeler transform (block sorting preprocessing that makes compression easier)
- Prediction by partial matching (PPM)
Lossy Compression
- Discrete cosine transform based coders
  - MPEG (the founder of video compression and still in use today, uses DCT and delta)
  - MP3 (sound and music compression)
  - JPEG (image compression using a windowed cosine transform, then quantization, then Huffman coding)
- Fractal compression
  - Fractal transform
- Wavelet compression

The Lempel-Ziv (LZ) compression methods are the most popular algorithms for lossless storage. DEFLATE is a variation on LZ which is optimized for decompression speed and compression ratio. Compression can be slow. DEFLATE is used in PKZIP, gzip and PNG. LZW (Lempel-Ziv-Welch) was patented by Unisys until June of 2003, and is used in GIF images. This patent is the main reason for GIF's increasing obsolescence. Also noteworthy are the LZR (LZ-Renau) methods, which serve as the basis of the Zip method. LZ methods utilize a table based compression model where table entries are subsitituted for redundant data. For most LZ methods, this table is generated dynamically from earlier data in the input. The table itself is often Huffman encoded (eg. SHRI, LZX). The current LZ based code that performs best is the obsolete LZX, although RAR and ACE are now coming close. LZX was purchased by Microsoft, slightly reduced in potency, and used in the CAB format.

Compression of sounds is generally called audio compression, where methods of psychoacoustics are used to remove non-audible components of the signal to make compression more efficient. Audio compression is therefore lossy compression. Different audio compression standards are listed under audio codecs.

External Links

Public domain article on data compression

In the News

[Scary] Pregnant woman says 'maternal instinct' helped her kill attack
FORT MITCHELL, Ky. - A pregnant woman who killed her attacker said a maternal instinct helped her fight off the woman who investigators believe was after her unborn child."I do believe that I fought harder because it was for my child,"Sarah Brady told ABC's "Good Morning America"in interviews aired Sunday and Monday. "It is a maternal instinct to protect your child to the very end."Katherine Smith, 22, died Thursday after luring Brady to her apartment to pick up a package supposedly delivered to the wrong address. When Smith pulled out a knife and attacked the pregnant woman, Brady fought back, striking Smith on the head with an ash tray and stabbing her three times with her own knife, police said. Brady, 26, said she didn't know Smith before the two met at Smith's apartment and can't be certain why Smith wanted to kill her."I really am not sure what was going through her mind,"Brady told ABC. "The only thing I thought was that she was going to kill me and my child and that is the only thing that ran through my mind."

The Boom in Bomb Detection: Get Ready to Be Scanned, Sniffed and Zappe
May 2004 article that describes technological advances in the area of bomb detection when materials are carried in cars, packages, or on a person. Methods discussed include X-rays, "the quadruple resonance technology that zaps people with low-frequency radio waves,"and "electronic sniffers."From Scientific American.

College Admissions: Study quantifies minority enrollment losses if aff
A nationwide ban on affirmative action in college admissions would cause a 10 percent drop in black and Hispanic enrollment at the nation's most selective colleges and universities, according to a new study.

Some Biofuels Are Worse Environmentally Than Fossil Fuels, Analysis Sh
Biofuels reduce greenhouse-gas emissions in comparison to fossil fuels. In the journal Science, researchers consider environmental costs of biofuel production. Corn, soy and sugarcane come up short. The authors urge governments to be far more selective about which biofuels they support, as not all are more environmentally friendly than fossil fuels.

Bugs, Even The 'Bad' Ones, Can Be Educationally Beneficial, New Book S
We have much to learn from bad bugs, according to Gilbert Waldbauer, whose book "Insights From Insects: What Bad Bugs Can Teach Us"was published today (Prometheus Books).

January 2005 California Landslide Information
This site features maps that aim to forecast rainfall and areas susceptible to landslides in California. Also provides a FAQ for 2005 Southern California landslides, a FAQ about landslides in general, a fact sheet on landslide types and processes, and related information. From the U.S. Geological Survey (USGS).

Tiny Pills Of RNA Fed To Planarians Help Researchers Identify Genes Es
University of Utah researchers--feeding microscopic pills of RNA to quarter-inch long worms called planarians--have identified many genes essential to understanding a biological mystery that has captivated scientists for hundreds of years: regeneration.

Gene Found In 90 Percent Of Breast Cancers May Be Cancer Vaccine Targe
A gene that appears to help regulate normal embryonic development is found at high levels in virtually all forms of breast cancer, according to a new study led by Laszlo Radvanyi, Ph.D., an associate professor of breast and melanoma medical oncology at The University of Texas M. D. Anderson Cancer Center.

Server Farms Live Off Open Source
The most visited websites on the internet run on open source software. But while the sites are ratcheting up page views, the companies behind them are less gung-ho about releasing their own code. Joanna Glasner reports from the O'Reilly Open Source Convention in Portland, Oregon.

MP3 Music Downloads

Preview songs, Download Free Music,Burn CDs at ITunes.com

Data compression

External Links

In the News

MP3 Music Downloads

Movies