Archiving files in Linux. What is a gzip file and how do I open a gzip file? Gzip opening working with text documents
gzip (short for GNU zip) is a computer file compression and recovery (decompression) utility that uses the DEFLATE algorithm. It is used mainly in UNIX systems, in some of which it is the de facto standard for data compression. It was created by Jean-loup Gailly and Mark Adler. Version 0.1 was first released on October 31, 1992, and version 1.0 was released in February 1993.
In accordance with the traditions of Unix programming, gzip performs only two functions: compression and decompression of a single file; it cannot pack multiple files into one archive. When compressed, the suffix .gz is added to the original file extension. To package multiple files, they are usually first archived into one file using the tar utility, and then this file is compressed with gzip. Thus, compressed archives usually have the double extension .tar.gz or the shortened .tgz.
On the other hand, this feature gives gzip the ability to work with a continuous stream of data, packing/unpacking it on the fly. This is widely used in UNIX systems: with the help of stream redirection, you can work with packed files as easily as with unpacked ones (unpacking them in memory when reading and packing them when writing); many Unix utilities have built-in support for this mechanism. Recently, gzip has been actively used to compress Internet traffic. gzip is now supported by most modern browsers.
In addition, there is a set of utilities for searching, displaying and comparing data in gzip format: zcat, zdiff, zfgrep, zless, zcmp, zegrep, zgrep, zmore.
- The computer determines the file type by its extension.
- By Windows default does not show file name extensions.
- Some characters cannot be used in the file name and extension.
- Not all formats are related to the same program.
- Below are all the programs that can be used to open the GZIP file.
HaoZip- Chinese clone the popular Winrar archiver, both in terms of functionality and interface as a whole. The archiver can work with all popular formats, including 7Z, ZIP, TAR, RAR, ISO, UDF, ACE, UUE, CAB, BZIP2, ARJ, JAR, LZH, RPM, Z, LZMA, NSIS, DEB, XAR, CPIO, SPLIT, WIM, IMG and others. In addition, using Haozip you can mount ISO images and view pictures through the built-in viewer, which is a very useful feature for archivers. As for the interface, the Chinese developers have done a good job here. They not only copied the design and functionality from the Winrar archiver, but also added...
WinRAR famous program, designed for working with archives. The utility includes a wide range of built-in capabilities. WinRAR compresses data faster than its competitors, saving disk space and user time. Supports well-known archive formats and is suitable for compressing multimedia files. Automatic file format recognition, a special data compression algorithm and an optimal packaging method are the advantages of the application. WinRAR can compress executive, multimedia files and object module libraries. The application allows you to divide archives into separate volumes and save them on different drives information.
When developing the FreeArc archiver, the author decided to create a program that compresses files at maximum speed. This required best qualities compression libraries LZMA, PPMD and GRZipLib. During the packaging process, the archiver forms files by type and performs compression using the most appropriate algorithm. When working, the archiver uses more than ten different algorithms and filters. If you compare this with common archivers, then 7-zip has only three, and RAR uses only seven algorithms. The archiver is easily adaptable for installation on various systems. It is developed on an open platform giving...
7-Zip is a well-known open source archiver. source code. This feature allows you to make changes to the structure of the program, adding certain functions to it. The program is clear and simple interface and has unique algorithms that speed up data archiving and unpacking. Also, this program can perform standard operations with the archive, for example, you can set a password for the file, or set the compression level of the archive. Also, if necessary, you can create a self-extracting archive with the necessary parameters, which are specified in special comments to the archive.
ExtractNow is convenient program, which allows you to unpack archived files quickly: just by pressing one button. This option will be especially convenient for those users who regularly have to unpack many files. The only negative is that the program does not support creating archives, because... is exclusively an unpacker (high-quality and convenient), and not an archiver. To unpack a file, you need to drag the archives into the program window and click the Extract button. Supports popular archive formats. Thus, the program can unpack all the popular and most frequently used...
Simplyzip is a convenient archiver with all the necessary functions that most users use. The program works with almost all popular archive formats, including rar or zip. However, due to the fact that the developers winRar programs do not allow the use of algorithms of their format; Rar archives can only be unpacked or their contents viewed. However, Simplyzip supports the installation of various modules and plugins that can expand the functionality of this archiver. If you install the necessary plugin, the program can be taught to create both Rar archives and archives of other formats...
Every year, to improve and enhance the convenience of users, developers of various computer software release many interesting, convenient and high-quality programs that significantly help in their work. One of these programs is the latest development of the WinArc archiver. This utility is available for public use and with its help you can easily archive or unarchive any file that interests you. During operation, this utility uses all processor cores, thereby speeding up the archiving process. In addition, it has the function of adjusting the use of RAM, which will significantly reduce the load on your computer...
There comes a time in every man’s life when traffic grows and the server dies, and you need to think about optimization. In the latest PHP Digest (No. 40), the article “How GZIP Compression Works” was mentioned with a link. Based on statistics, 56% of websites use GZIP. I hope this article will highlight the benefits of this technology to the reader.
There may be errors in the text (I proofread it several times, but still suddenly), so I apologize in advance and ask you to inform me about any problems via private messages if any part of the translation seems incorrect to you.
Even in modern world, with high-speed Internet connections and unlimited information storage, data compression is still relevant, especially for mobile devices and countries with slow Internet connections. This post describes the de facto lossless compression method for compressing text data on websites: GZIP.
GZIP compression
GZIP provides lossless compression, meaning the original data can be fully recovered when decompressed. It is based on the DEFLATE algorithm, which uses a combination of the LZ77 algorithm and the Huffman algorithm.LZ77 algorithm
The LZ77 algorithm replaces repeated occurrences of data with “links”. Those. If in the available data some chain of elements occurs more than once, then all subsequent occurrences of it are replaced with “links” to its first instance. The algorithm is perfectly reviewed and described. Each such link has two values: offset and length.Let's look at an example:
Original text: “ServerGrove, the PHP hosting company, provides hosting solutions for PHP projects” (81 bytes)
LZ77: "ServerGrove, the PHP hosting company, p<3,32>ides<9,26>solutions for<5,52><3,35>jects" (73 bytes, assuming that each reference is 3 bytes)
As you may have noticed, the words "hosting" and "PHP" are repeated, so the second time the substring is found it will be replaced with a link. There are other matches, such as "er", but... it's insignificant (in this case - “er” is missing in other words), the original text remains.
Huffman coding
Huffman coding is a variable-length encoding method that assigns shorter codes to more frequent "characters". The problem with variable code length is usually that we need a way to know when a code has ended and a new one has started in order to decipher it.Huffman coding solves this problem by creating a prefix code where no codeword is a prefix of another. This may be clearer with an example:
>Original text: "ServerGrove"
ASCII codification: "01010011 01100101 01110010 01110110 01100101 01110010 01000111 01110010 01101111 01110110 01100101" (88 bits)
ASCII is a fixed-length character encoding system such that the letter "e", which is repeated three times, is also the most common letter in English language, is the same size as the letter "G", which appears only once. Using this statistical information, Huffman can create the most optimized system
Huffman: "1110 00 01 10 00 01 1111 01 110 10 00" (27 bits)
Huffman's method allows us to obtain shorter codes for "e", "r" and "v", while producing longer "S" and "G". Explaining how to use the Huffman method is beyond the scope of this post, but if you're interested, I encourage you to check out the excellent video at computerphile (or article on Habré).
DEFLATE as an algorithm that is used in GZIP compression is a combination of both of these algorithms.
Is GZIP the best compression method?
The answer is no. There are other methods that give better compression rates, but there are several good reasons to use this one.First, even though GZIP is not the best compression method, it provides a good compromise between speed and compression ratio. GZIP compresses and decompresses quickly and the compression ratio is high.
Secondly, it is not easy to implement a new global data compression method that everyone can use. Browsers will need to be updated, which is much easier today with auto-updates. However, browsers are not the only problem. Chromium tried to add support for BZIP2, more best method based on the Burrows-Wheeler transformation, but it had to be abandoned because some intermediate proxy servers distorted the data because... could not recognize bzip2 headers and tried to process gzip content. Bug report is available.
GZIP + HTTP
The process of receiving compressed content between the client (browser) and the server is quite simple. If the browser has GZIP/DEFLATE support, it lets the server know this through the “Accept-Encoding” header. Then, the server can choose to send the content in compressed or original form.
Implementation
The DEFLATE specification provides some freedom for developers to implement the algorithm using different approaches, as long as the resulting stream is compatible with the specification.GNU GZIP
The GNU implementation is the most common and was designed to be a replacement for the archiving utility, free of proprietary algorithms. To compress a file using the GNU GZIP utility:$ gzip -c file.txt > file.txt.gz
There are 9 compression levels, from "1" (fastest with the lowest compression ratio) to "9" (slowest with the best compression ratio). The default is "6". If you need maximum compression at the expense of using more memory and time, use the "-9" (or "-best") flag:
$ gzip -9 -c file.txt > file.txt.gz
7-zip
7-zip implements the DELFATE algorithm differently and usually archives from b O higher compression ratio. To compress a file as much as possible:7z a -mx9 file.txt.gz file.txt
7-zip is also available for Windows and provides implementations for other compression methods such as 7z, xz, bzip2, zip and others.
Zopfli
Zopfli is ideal for one-time compression, such as in situations where a file is compressed once and reused. It is 100 times slower, but compression is 5% better than others. Habrapost.Enabling GZIP
Apache
The mod_deflate module provides GZIP support so that the server response is compressed on the fly before it is sent to the client over the network. To enable compression of text files, you need to add the following lines to .htaccess:AddOutputFilterByType DEFLATE text/plain
AddOutputFilterByType DEFLATE text/html
AddOutputFilterByType DEFLATE text/xml
AddOutputFilterByType DEFLATE text/css
AddOutputFilterByType DEFLATE application/xml
AddOutputFilterByType DEFLATE application/xhtml+xml
AddOutputFilterByType DEFLATE application/rss+xml
AddOutputFilterByType DEFLATE application/javascript
AddOutputFilterByType DEFLATE application/x-javascript
There are several known bugs in some browser versions, so it is recommended* to also add:
BrowserMatch ^Mozilla/4 gzip-only-text/html*this solution is currently no longer relevant, like the above browsers, so this information can be taken for informational purposes
BrowserMatch ^Mozilla/4\.0 no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
Header append Vary User-Agent
Additionally, you can use pre-compressed files instead of compressing them each time. This is especially useful for files that do not change with every request, such as CSS and JavaScript, which may be compressed using slow algorithms. For this:
RewriteEngine On
AddEncoding gzip .gz
RewriteCond %(HTTP:Accept-encoding) gzip
RewriteCond %(REQUEST_FILENAME).gz -f
RewriteRule ^(.*)$ $1.gz
This tells Apache that files with a .gz extension should be provided compressed (line 2), it should check whether the browser can accept gzip (line 3), and if a compressed file exists (line 4), we add the .gz for the requested file.
Nginx
The ngx_http_gzip_module allows you to compress files using GZIP on the fly, while the ngx_http_gzip_static_module allows you to send pre-compressed files with a “.gz” extension instead of regular ones.An example configuration looks like this:
gzip on;
gzip_min_length 1000;
gzip_types text/plain application/xml;
GZIP + PHP
Although compressing data using PHP is generally not recommended as it is quite slow, it can be done using the zlib module. For example, we use maximum compression using the jQuery.min library:$originalFile = __DIR__ . "/jquery-1.11.0.min.js"; $gzipFile = __DIR__ . "/jquery-1.11.0.min.js.gz"; $originalData = file_get_contents($originalFile); $gzipData = gzencode($originalData, 9); file_put_contents($gzipFile, $gzipData); var_dump(filesize($originalFile)); // int(96380) var_dump(filesize($gzipFile)); // int(33305)
Instead of output (translator's note)
Despite how hands itched to add the author’s own explanations of algorithms, statistics and comparison test results to the article, the translation was completed practically without interference from the translator. The translation of the article was carried out with the permission of the author and the ServerGrove portal.Comparison of archivers in Linux
For a long time I wanted to spend comparison test popular archivers that are available in Linux and finally I had some free time to do it. I did not test exotic or outdated archivers; I took part in the experiment 7z, xz, zip, bzip2, gzip and even downloaded a trial version of the archiver rar. All archivers were tested with three various settings compression - fastest, average (default) and best compression. I’ll say right away that I did not experiment with additional options like the size of dictionaries - I left all this by default, otherwise the comparison of archivers would have been extended several times and would have become less clear. Moreover, various tricks with compression options do not always lead to the expected results; the developers know their creations quite well, so in order to achieve maximum data compression in 7z, just use the option -mx=9 and don't worry about it :)Now I have several thick files with text open in front of me, from which I need to assemble an article, without forgetting anything necessary and throwing out unnecessary rubbish. This is the most difficult thing :) I will try to present all this in a more or less readable form. Go!
Tested archivers and unpackers
How was the execution time measured?
The time utility. This utility can show the actual number of seconds spent by the processor to complete a task. This is very important for testing archivers. For example, if I decide to compress the Linux kernel sources ZIP archiver and I will time the task completion time with a regular stopwatch, I will find that the first time the compression took about 2 minutes, the second time - about 30 seconds, the third time - 10 seconds. I think there is no need to say why this happens. The time utility allows you to avoid such problems and in all cases displays the same result - the number of seconds spent by the CPU to complete the task (Total number of CPU-seconds that the process spent in user mode.) Roughly speaking, the test results do not depend on the reading speed data from the hard drive.
What types of files were compressed?
In what environment were the tests conducted?
32-bit CPU, 4GB DDR2 RAM. Gentoo Linux distribution, file system ext4fs. All tests were carried out in the console with the Xs turned off, unnecessary demons stopped, and in general everything that was possible was turned off, even the light :) That is. the computer was used exclusively for testing archivers and nothing more.
What was taken into account in testing
A small educational program.
LZMA - new algorithm data compression (compression), which is used to create archives in 7z format. Features: high compression ratio, use of dictionaries of various sizes, open source code. - open source utility and algorithm for data compression. Following the Unix ideology, bzip2 performs only one task - compressing a file. To archive multiple files or directories, it must be used together with the utility TAR. Tar archives files into one, and bzip2 compresses them. That’s why the extension for such archives is .tar.bz2
Gzip- a utility and algorithm for data compression and decompression. Just like bzip2, it performs only one task - compresses or decompresses one file. Has a high operating speed.
Deflate- data compression algorithm.
TAR- (tape archive, archive on magnetic tape) - archive format, utility for working with archives of the same name. Tar does not compress data, it creates one file out of many, which is usually then archived using the gzip or bzip2 algorithm.
RAR- Paid popular data compression format and program for working with various types archives. Author - Evgeniy Roshal.
Unrar- free unpacker RAR archive ov.
ZIP- Standard, popular data compression format.
XZ- Data compression format that uses the LZMA compression algorithm. Created as a replacement for lzma. XZ-utils is an implementation of the XZ format on Linux. The TAR archiver, starting from version 1.22, allows you to create archives in xz format.
7z- data compression format using the LZMA compression algorithm.
p7zip- implementation of 7-Zip archiver for Linux.
7-Zip- a file archiver that supports many compression algorithms. Author - Igor Pavlov. The main format is 7z using the LZMA algorithm.
q7z- GUI for p7zip archiver
What options were the archivers launched with?
| Create a RAR archive. Default compression. | rar a -r archive.rar files |
| Create a RAR archive. Fast compression | rar a -r -m1 archive.rar files |
| Create a RAR archive. Maximum compression | rar a -r -m5 archive.rar files |
| Create ZIP archive. Default compression | zip -r archive.zip files |
| Create a ZIP archive. Fast compression | zip -r -1 archive.zip files |
| Create a ZIP archive. Maximum compression | zip -r -9 archive.zip files |
| Create a 7z archive. Default compression | 7z a archive.7z files |
| Create a 7z archive. Fast compression | 7z a -mx=1 archive.7z files |
| Create a 7z archive. Maximum compression | 7z a -mx=9 archive.7z files |
| Create a bzip2 archive. Maximum (default) compression | tar cfj archive.tar.bz2 files |
| Create a bzip2 archive. Medium compression | tar cf archive.tar files bzip2 -vc4 archive.tar > archive.tar.bz2 |
| Create a bzip2 archive. Fast compression | tar cf archive.tar files bzip2 -vc1 archive.tar > archive.tar.bz2 |
| Create a gzip archive. Default compression | tar cfz archive.tar.gz files |
| Create a gzip archive. Maximum compression | tar cf archive.tar files gzip -vc9 archive.tar > archive.tar.gz |
| Create a gzip archive. Fast compression | tar cf archive.tar files gzip -vc1 archive.tar > archive.tar.gz |
| Create XZ archive. Default compression | export XZ_OPT="" |
| Create XZ archive. Fast compression | export XZ_OPT="-1" tar cfJ archive.tar.xz files |
| Create XZ archive. Maximum compression | export XZ_OPT="-9 --extreme" tar cfJ archive.tar.xz files |
Add. info
To measure time, the time utility was used: % /usr/bin/time -f "%U" Bzip2 by default archives in the maximum compression mode (9), so the default compression was taken as the best, and compression 4 was used as the average (1 - fastest, 9 - best).
In some tests I decided to use ZIP, RAR and 7z along with the TAR archiver. You will see the results :)
The results for the 7z and XZ are noticeably different. Although 7z uses LZMA, just like XZ, I tested them as separate archivers.
To create XZ archives, TAR 1.21 uses lzma, and TAR 1.22 uses XZ.
To have TAR automatically detect the archive type, use the key a: % tar xaf archive.tar.bz2 When creating XZ archives in TAR, compression options can be specified via environment variable XZ_OPT: % export XZ_OPT="-9 --extreme"
Test No. 1. Compressing media files.
For the 1st test, I collected a large pile of files that were essentially already compressed. These are mp3, wav, jpg, png, pdf, avi. They all compress almost equally badly :)As a result, I ended up with a catalog full of all sorts of goodies:
At maximum compression, RAR has the best results - it compressed data unexpectedly better and faster than others. Although you would hardly want to spend 1029 seconds to save 40MB. Gzip and ZIP did not compress as well, but they were 10 times faster than RAR and XZ.
At medium compression, RAR wins in the “slow, but good” category, while Gzip and ZIP win in the “not so good, but very fast” category. Bzip2 has an average result - it seems to compress quite quickly, but only slightly better than Gzip.
In fast mode, the size of the archives turned out to be almost the same, but the time was not quite the same. The clear leader here is regular ZIP, which compressed data just as well as gzip and rar, but did it much faster. In general, the conclusion is this: if you compress media files, i.e. files that are already compressed and you are short on time - use Gzip or Zip. If you have enough time - Rar or 7z/xz.
Bzip2, by the way, took a very long time to unpack its archives compared to others, and RAR probably has the best result in terms of speed/quality.
Test No. 2. Binary file compression
In the 2nd test, binary files will be compressed. I just copied the heaviest ones from my /usr/bin/ directory.Available: 100 binary files with a total size of 167MB
At maximum compression, the 7z performed much better than the XZ - it compressed faster and better. Zip turned out to be the fastest - faster than Gzip with same sizes archives.
In the medium compression mode everything is the same, but there is no point in using Bzip2 - it compresses only slightly better than Gzip/ZIP, but is noticeably slower.
In fast mode, the 7z was again better than the XZ. (Why? They both use LZMA!)
By the way, 7z in fast mode produced an archive of 56MB in 35 seconds, and Bzip2 in maximum (default) compression mode produced an archive of 64MB in 65 seconds. Those. 7z outperformed Bzip2 in all respects :)
Test No. 3. Compressing a CD image
Separately, for some reason, I decided to compress the CD image. It can be safely attributed to the content of test No. 1, but I decided to put it separately, just in case :)Available: 1 CD image size 614MB
In the best compression mode, 7z did its job slightly worse, but faster than XZ, but RAR turned out to be the slowest. I already have an impression about Bzip2 - it seems to compress much faster than rar/7z/xz, but much slower than gzip/zip, and the archive size is not much better than gzip/zip. If I want to compress something well and I have time for it, I will most likely choose 7z. If I need to compress files as quickly as possible, I will choose Gzip or ZIP. Gzip, by the way, has a big bonus over ZIP - it saves all information about file owners and rights, which ZIP does not do (like 7z, by the way, but more on that later).
At medium settings, the size of the archives (as well as the time) turned out to be almost identical to the previous ones.
But in the fast compression mode, RAR pleasantly surprised me - it compressed faster and better than all the others. Bzip2 in fast mode compressed even worse than Gzip/ZIP. Bzip2 spent much more time unpacking the archive than all other archivers. And XZ unpacked the archive noticeably faster than 7z.
Test No. 4. Compressing Linux Kernel Sources
One of standard methods check the quality of the data compression algorithm - kernel sources.Available:
Sources linux-2.6.33. This is about 31 thousand files with a total volume of 432MB.
Since there are a lot of files, I used the solid (continuous archive) option of RAR. 7z creates a solid archive by default. It was also decided to use tar+rar and tar+zip. The results, by the way, are unexpected :)
Maximum compression mode: RAR has good results. True, with the TAR+RAR option, the archives weigh less than with RAR with the "solid" option =)
Average compression: Bzip2 suddenly performed better than RAR, vastly outperforming it in speed and compressing almost as well.
Fast compression mode: Here Bzip2 again compressed better than RAR, although it (Bzip2) did it the slowest. The best results in my opinion are with 7z.
Unpacking archives: The archives are compressed to the maximum, RAR took a very long time to unpack compared to others. 7z/XZ unpacked 10 times faster than RAR, and Gzip was just lightning fast.
Test No. 5. Uncompressed documents
Text documents such as TXT, ODF, DOC. I assume that the results will be similar to test No. 4 :)Available:
Mostly DOC files. I don’t store documents on this computer, so I had to download them from the Internet, and ODF is more difficult to find. Also several heavy TXT dictionary files, RTF and HTML files. A total of 66MB.
Best compression mode: RAR did the best. Bzip2, as always, is somewhere in the middle.
Medium compression mode: 7z/XZ files were the best compressed, but they took a long time to do so. Bzip2 compressed much faster and almost as good as RAR
Fastest compression mode: First time Bzip2 compressed better (though not faster) than 7z/XZ.
Unpacking archives: An archive compressed in maximum mode took a very long time to unpack from RAR. Actually, this is not the first time. Well, Bzip2, as always, is not the fastest for unpacking archives.
Test No. 6. Huge number of small text files - Gentoo Portage
Available: 116 thousand files with a total volume of 600MB.
Maximum compression mode: The smallest archive was obtained with 7z, and in terms of time it completed it 2 times faster than XZ. Bzip2 probably has the best result in terms of speed/compression
Medium compression mode: Bzip2, in my opinion, has better results than RAR. The best compression is still 7z.
Fast compression: XZ has the best result. 7z compressed 6MB better than XZ, but also 21sec slower. Unpacking the archives: I think XZ did the best job here. RAR, as always, is slow to unpack archives that are maximally compressed.
So, let's summarize. Who is the best archiver in the world?
I will not give an answer to this question :P I will say one thing - if compression speed is important to you, use Gzip, if you need to compress files as much as possible, use XZ or 7z. Personally, I would choose 7z, because... Basically it archives slightly faster than XZ. But, do not forget that archivers such as RAR, ZIP and 7z do not save complete information about file owners, rights, etc. You can specify the appropriate settings for RAR when creating and unpacking an archive, but it’s easier to just create a TAR archive and compress it. The manual for 7z directly states that to backup the system, first create a TAR archive, and then compress it into 7z.
It was fun to see how the TAR+RAR combination often works better than RAR alone :) In general, TAR creates one file (archive) from many exactly as quickly as your HDD, while hardly straining the processor.
Gzip or ZIP? Gzip! The results are the same, but Gzip saves all Additional information about files.
XZ or 7z? If you don't need to save file ownership information, use 7z. If necessary, create XZ archives via TAR.
RAR?. To be honest, I expected worse results from it, but RAR performed well in principle. Unless it's paid ;)
Bzip2?. This archiver gave me ambivalent feelings. On the one hand, it compresses faster than XZ/7z/RAR, but on the other hand, it compresses only slightly better than Gzip/ZIP and does so noticeably slower than the latter. In some tests it performed well, in others it compressed slower and worse than 7z at the most quick settings. It is precisely because of this inconsistency that I personally will refuse to use it in favor of 7z/XZ.
But ultimately it's up to you. Good luck!
Perhaps these materials will interest you.
