home > Internet > Create a local ubuntu repository. Create a local repository from deb packages

Create a local ubuntu repository. Create a local repository from deb packages

As you know, Ubuntu implements a centralized storage of information about all the main programs in one place, for easier search and subsequent installation. That place is the Synaptic package manager. (System -> Administration -> Synaptic Package Manager) To install the program, you only need to find it in the list and mark it for installation, then the program will be automatically downloaded from the Internet and installed.

The main programs that may come in handy when working in the Ubuntu OS are stored in the so-called repository on the official Ubuntu server. Every time you mark a program for installation in the Synaptic package manager, it is downloaded first.

But what if you have a very slow or expensive Internet, or maybe you don’t have access to the World Wide Web at all, or you just don’t want to depend on an Internet connection to work in linux ubuntu? Of course you can download everything. necessary programs(packages) one by one, and then manually install, but who knows what program will come in handy in the future?

The solution to this problem is to create a local repository, i.e. a full or partial copy of the official repository on your hard drive. Thus, once downloading the entire repository, you will no longer be dependent on the Internet.

So let's get started.

1. To create a copy (slice) of the repository on your hard drive, there is special program:debmirror. Accordingly, first you need to install it from the same Synaptic package manager

or by typing the command in the terminal:

sudo apt-get install debmirror

2. Create a folder in the home directory that will store a slice of the repository (or even several), for example, let's call it Repositories.

Let's go to the Repositories folder:

and create a script file in it called archive.ubuntulinux.org.sh:

gedit archive.ubuntulinux.org.sh

Paste the following into this file:

#!/bin/bash -x
/usr/bin/debmirror --nosource -m --passive --host=archive.ubuntulinux.org \
--root=ubuntu --method=ftp --progress \
--dist=hardy,hardy-security,hardy-updates,hardy-backports \
--ignore-release-gpg --section=main,restricted,multiverse,universe \
--arch=i386 /full/path/to/folder/where/to/download/repository/

your full path should look something like this: /home/aidsoid/Repositories/en.archive.ubuntu.com/

and save it. After saving, you need to make the file executable (chmod +x archive.ubuntulinux.org.sh).

The above example will fetch the repository for Ubuntu 8.04 (Hardy Heron), if you are still using the old version of Ubuntu 7.10 (Gutsy Gibbon) then you need to tweak this script a bit, in particular, you need to change the --dist parameter from hardy to gutsy . You should also pay attention to the --arch parameter, which indicates the architecture of your computer, the i386 value indicates that packages for an x86 computer will be downloaded. This parameter can also be set to amd64 for 64-bit AMD or Intel computers.

4. Well, that's all, now you can run the script for execution, for this, in the console, while in the Repositories folder, type the following command:

./archive.ubuntulinux.org.sh

The script will start running. The entire repository takes up about 20-30 gigabytes, so make sure you have free disk space before downloading. At any time, you can painlessly interrupt the script by pressing the Ctrl + C key combination. When you run the script again, it will continue downloading incomplete packages. Also, running the script again will update your local repository, i.e. the presence of new versions of programs is checked and they are also downloaded.

Note: Ubuntu 7.10 users have one small pitfall, Ubuntu users 8.04 is nothing to worry about. There is a bug in debmirror that causes the pool folder to be deleted from the local repository when the script is run again. It must be corrected before running the script. Bug description here: https://bugs.launchpad.net/ubuntu/+source/debmirror/+bug/136634

The error is easily fixed:
$ sudo gedit /usr/bin/debmirror
replace the line local $/="\n\n"; to local $/="\n"; and save the file. Now the update of the repository will pass without incident.

5. After the script has downloaded (this may take a long time), you can use it by adding a deb line to the application sources. (System -> Administration -> Application Sources)

deb line should point to the folder with the repository and looks something like this:
deb file:///home/aidsoid/Repositories/en.archive.ubuntu.com/ hardy main restricted multiverse universe

The gradual transition of the enterprise to GNU / Linux generates the need for appropriate changes in the infrastructure. Today we are solving the problem of updating client machines globally by creating a local repository. The process was originally documented as a reminder for the future, so I apologize in advance for possible absurdities in the text. So.
First you need to decide what is the best way to do this. The internets single out two favorites rsync and debmirror. I chose the latter because of its greater flexibility.

1. Getting the keys

To create a repository mirror, you need to get the "Ubuntu Archive Automatic Signing Key ". To do this, in the terminal from the superuser, enter:
gpg --no-default-keyring --keyring trustedkeys.gpg --recv-keys 437D05B5

2. Space preparation

Create a folder for the repository:
sudo mkdir /path/to/repository
Important! Try to keep track of free space in the specified path. Even two architectures i386 and amd64 take up a good amount of it.

3. Receiving packages

Mirroring takes place in three stages:

Removing unknown files (disabled with the option --nocleanup below);

Building a list of index archives and checking for presence in the local repository.To implement the above, we will create a file repo_update.sh with the following content.

#!/bin/sh
#This is our repository configuration. Depending on the parameters specified
#here, we will get the content we need.

#option cleanup. Enabled by default. After downloading the packages, it removes the earlier ones.
#versions. To disable the option, the --nocleanup parameter is required
clean=--nocleanup
#Source option. Downloads package sources. If you are not using
#source codes to study and modify applications (which is typical for
#binary distributions), feel free to set the --no-source option
src=--source

#host. The name of the server where we get the packets from.
servername=mirror.yandex.ru

#Root. The root directory on the server of our choice.
rdir=/ubuntu

#Ubuntu release name. Settings for 10.04 version.
release=lucid,lucid-backports,lucid-proposed,lucid-security,lucid-updates

#Sections.
section=main,restricted,universe,multiverse

# Synchronization protocol. Debmirror supports the following methods: http,
#hftp, ftp, rsync
sync_protocol=rsync

#Architecture. If only 32 or 64 bit systems are used.
#One of the architectures can be removed. Also, if other architectures are used,
#they should be added.
arch=i386,amd64

#Repository location. Specify the local folder created. in paragraph 2.
path=/path/to/repository

Debmirror --progress --verbose $clean $src --md5sums --host=$servername --root=$rdir \
--dist=$release -s=$section --method=$sync_protocol -a=$arch $path

Now let's put it in the directory /usr/local/bin and make it executable.
chmod +x repo_update.sh
sudo cp repo_update.sh /usr/local/bin/

Next, run the resulting script and wait for the process to complete. The process is quite long. The execution time is highly dependent on your internet bandwidth.
sudo /usr/local/bin/repo_update.sh
Attention! The download size exceeds tens of gigabytes, and the government Internet is rarely unlimited. Furthermore, debmirror sensitive to connection stability, 120 seconds of downtime and you have to start all over again.

4. Web server setup

In order not to make unnecessary dances with a tambourine, we will choose a protocol http, as the traditional method of granting access to a repository. The choice of the web server is up to you. From favorites ngnix, apache and lighttpd, I chose the latter due to the lack of experience with it (pleasant with useful, yes). So.

Server installation.

sudo apt-get install lighttpd
Everything is simple here. If you do not plan to use as www directory other than the default, the server does not need to be configured. All you have to do is create a symbolic link in the directory /var/www
ln -s /path/to/repository /var/www/ubuntu

Let's check the availability of the repository from the browser: http:// /ubuntu/

5. Client setup

Here we apply a little trick. In order not to make changes to /etc/apt/sources.list (you never know what happens). Add to file /etc/hosts a couple of lines.
en.archive.ubuntu.com
security.ubuntu.com
Note. In the presence of DNS servers you can write all this in it, and write the true addresses of the above names on the repository server.

6. Automation

And now for the sweetest part. Let's make it all spin on its own.

6.1 Back end

In point #3, we created a script with which we received packages. Let's configure its autorun using the daemon tools cron.
sudo crontab -e

In which we add the treasured line:

0 0 * * * /usr/local/bin/repo_update.sh
Now every day at 0:00 our script will do all the routine work for us.

6.2 Client side

Let's create a script on clients system_upd.sh in the directory /usr/local/bin the following content:
#!/bin/sh
apt-get -y update && apt-get -y upgrade && apt-get -y clean

Let's not forget to make it executable.
sudo chmod +x /usr/local/bin/system_upd.sh

Then we open cron:
sudo crontab -e

And add the line:
40 17 * * * /usr/local/bin/system_upd.sh

Now every day at 17:40 the system will poll our repository for updates and update if any are found.

Attention! When working with crontab do not forget that after the lines with tasks there must be an empty line, which is indicated by the sign " # ".
p.s.: I apologize for the lack of attached images, but in this case I think their presence is simply inappropriate.

If ($answer_counter == 1): ?> endif; ?>

* Make offline repository via LAN *

Install local Apache web server >

# apt-get install apache2

By default, the Apache package will install the website in /var/www on your system. For our purposes, this is fine, so there's no reason to do anything else. You can easily test it by pointing your favorite browser to http://localhost . You should see the web page after the default install, which is actually stored in /var/www/index.html

Create a Debian Package Repository Directory

I chose the /var/www/debs directory for this. Under it, you must create "architecture" directories, one for each architecture that needs to be supported. If you're only using one computer (or type of computer), you'll only need one - usually "i386" for 32-bit systems, or "amd64" for 64-bit. If you are using any other architecture, I will assume that you probably already know about it. Now just copy the ".deb" package files for this architecture into the appropriate directories. If you now point your favorite web browser to http://localhost/debs/amd64 (for example), you will see a list of packages for 64-bit systems.

Create Packages.gz file

Now we need to create a directory file for APT to use. This is done with a utility called "dpkg-scanpackages". Here are the commands I use to update AMD64 packages in local network:

# cd /var/www/debs/ # dpkg-scanpackages amd64 | gzip -9c > amd64/Packages.gz

Make known APT repository

Now all that's left is to tell APT about your repository. You do this by updating the /etc/apt/sources.list file. You will need an entry like this one:

deb http://localhost/debs/amd64/

I used my system's actual hostname instead of localhost - that way the code will be the same for all computers on my local network, but localhost will work just fine if you're only using one computer.
Now update APT:

# apt-get update

Answered by user61928 on 09/05/2012 at 14:43

There are 4 steps to setting up a simple repository for yourself

1.Install dpkg-dev
2.Put the packages in a directory
3.Create a script that will scan the packages and create a file apt-get update can read
4. Add a line to your sources.list pointing at your repository

Install dpkg-dev

Type in a terminal
sudo apt-get install dpkg-dev
Create a directory where you will keep your packages. For this example, we"ll use /usr/local/mydebs.
sudo mkdir -p /usr/local/mydebs
Now move your packages into the directory you"ve just created.

Previously downloaded Packages are generally stored on your system in the /var/cache/apt/archives directory. If you have installed apt-cacher you will have additional packages stored in its /packages directory.

The Script update-mydebs

It's a simple three liner:
#! /bin/bash cd /usr/local/mydebs dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz
Cut and paste the above into gedit, and save it as update-mydebs in ~/bin. (the tilde "~" means your home directory. If ~/bin does not exist, create it: Ubuntu will put that directory in your PATH. It's a good place to put personal scripts). Next, make the script executable:
Chmod u+x ~/bin/update-mydebs How the script works:
dpkg-scanpackages looks at all the packages in mydebs, and the output is compressed and written to a file (Packages.gz) that apt-get update can read (see below for a reference that explains this in excruciating detail). /dev/null is an empty file; it is a substitute for an override file which holds some additional information about the packages, which in this case is not really needed. See deb-override(5) if you want to know about it.

Sources.list

Deb file:/usr/local/mydebs ./
to your /etc/apt/sources.list, and you"re done.

CD Option

You can burn the directory containing the debs to a CD and use that as a repository as well (good for sharing between computers). To use the CD as a repository, simply run
sudo apt-cdrom add
Using the Repository

Whenever you put a new deb in the mydebs directory, run
sudo update-mydebs sudo apt-get update
Now your local packages can be manipulated with Synaptic, aptitude and the apt commands: apt-get, apt-cache, etc. When you attempt to apt-get install, any dependencies will be resolved for you, as long as they can be met.

Badly made packages will probably fail, but you won't have endured dpkg hell.

Answered by BigSack on 08/16/2012 at 13:45

Create an authenticated repository

I've looked at the answers here and on other sites and most of them (a downside IMHO) are something you don't configure verified the authenticity of the repository. This means you need to run apt-get with --allow-unauthenticated to install packages from it. This can be a security risk, especially in scenarios where the packages you install may not all be from your local repository.

Note that I didn't cover here how to make it available on LAN, but it's a fairly common configuration using Apache or nginx (see other answers here).

Setting up a repo directory

Mkdir /home/srv/packages/local-xenial cd /home/srv/packages/local-xenial

Then add this line to sources.list:

Deb file:/home/srv/packages/local-xenial/ ./

Adding and removing packages

remove packages

Rm /home/srv/packages/local-xenial/some_package_idont_like

add packages

Cp /some/dir/apackage.deb /home/srv/packages/local-xenial

now the following script is run which generates the Packages, Release and InRelease files and signs them with private key gpg:

#!/bin/bash if [ -z "$1" ]; then echo -e "usage: "basename $0" DISTRO where DISTRO is the Ubuntu version codename (e.g. 14.04 is trusty)\n The way to use this script is to do the changes to the repo first, i.e. delete or copy in the . deb file to /srv/packages/local-DISTRO, and then run this script\n This script can be run as an unprivileged user - root is not needed so long as your user can write to the local repository directory" else cd /srv /packages/local-"$1" # Generate the Packages file dpkg-scanpackages . /dev/null > Packages gzip --keep --force -9 Packages # Generate the Release file cat conf/distributions > Release # The Date: field has the same format as the Debian package changelog entries, # that is, RFC 2822 with time zone +0000 echo -e "Date: "LANG=C date -Ru"" >> Release # Release must contain MD5 sums of all repository files (in a simple repo just the Packages and Packages.gz files) echo -e " MD5Sum:" >> Release printf " "$(md5sum Packages.gz | cut --delimiter=" " --fields=1)" %16d Packages.gz" $(wc --bytes Packages.gz | cut --delimiter =" " --fields=1) >> Release printf "\n "$(md5sum Packages | cut --delimiter=" " --fields=1)" %16d Packages" $(wc --bytes Packages | cut - -delimiter=" " --fields=1) >> Release # Release must contain SHA256 sums of all repository files (in a simple repo just the Packages and Packages.gz files) echo -e "\nSHA256:" >> Release printf " "$(sha256sum Packages.gz | cut --delimiter=" " --fields=1)" %16d Packages.gz" $(wc --bytes Packages.gz | cut --delimiter=" " --fields=1) >> Release printf "\n "$(sha256sum Packages | cut --delimiter=" " --fields=1)" %16d Packages" $(wc --bytes Packages | cut --delimiter=" " --fields=1) >> Release # Clearsign the Release file (that is , sign it without encrypting it) gpg --clearsign --digest-algo SHA512 --local-user $USER -o InRelease Release # Release.gpg only need for older apt versions # gpg -abs --digest-algo SHA512 -- local-user $USER -o Release.gpg Release # Get apt to see the changes sudo apt-get update fi

Sample contents of conf/distributions file

Origin: My_Local_Repo Label: My_Local_Repo Codename: xenial Architectures: i386 amd64 Components: main Description: My local APT repository SignWith: 12345ABC

Answered by happyskeptic on May 15, 2016 at 09:33

You can also set up a local origin server with nginx and reprepro:

Install debian packages

sudo apt-get install reprepro nginx

create directories for reprepro and edit it

sudo mkdir -p /srv/reprepro/ubuntu/(conf,dists,incoming,indices,logs,pool,project,tmp) $ cd /srv/reprepro/ubuntu/ $ sudo chown -R "whoami" . # changes the repository owner to the current user

/srv/reprepro/ubuntu/conf/distributions

Origin: Your Name Label: Your repository name Codename: karmic Architectures: i386 amd64 source Components: main Description: Description of repository you are creating SignWith: YOUR-KEY-ID

/srv/reprepro/ubuntu/conf/options

Ask-passphrase basedir .

Include it in reprepro, build it

$ reprepro includedeb karmic /path/to/my-package_0.1-1.deb \ # change /path/to/my-package_0.1-1.deb to the path to your package

nginx config:

/etc/nginx/sites-available/vhost-packages.conf

Server ( listen 80; server_name packages.internal; access_log /var/log/nginx/packages-access.log; error_log /var/log/nginx/packages-error.log; location / ( root /srv/reprepro; index index. html; ) location ~ /(.*)/conf ( deny all; ) location ~ /(.*)/db ( deny all; ) )

Optimize bucket size:

/etc/nginx/conf.d/server_names_hash_bucket_size.conf

server_names_hash_bucket_size 64;

Answered by elprup on 07/03/2013 at 11:18

There are several reasons you may want to create a local repository. The first is that you want to save on bandwidth if you have multiple Ubuntu machines to update. For example if you had 25 Ubuntu machines that all needed updating at least once a week, you would significantly save bandwidth because you could do all but the repository locally.

Most organizations have decent bandwidth for their network gateways but this bandwidth is a precious commodity that needs to be used wisely.

Many organizations still have routers with 10MB or 100MB limits at the gateway but 1 GB network connections internally so bandwidth could be better used internally. The second reason for creating your own repository is that you can control what applications are loaded on your internal Ubuntu machines.

You can remove any applications your organization does not want to use on the local network from the repository that updates the machines. Even better, you can create a test box and test applications and versions before you allow them to roll out into your network assuring security and stability.

You first have to set up a mirror, to do that you need to Just press Ctrl + Alt + T on your keyboard to open Terminal. When it opens, run the command below.
apt-get install apt-mirror
Once you have your set up apt-mirror you can start your download of the repository with this command.

apt-mirror /etc/apt/mirror.list 1

For system administrators this topic is almost of paramount importance. After all, usually any organization, taking care of the security and reliability of its servers and networks in general, develops and implements certain security policies. Which, in turn, provide for restrictions on access to open internet for most client machines from the local network. However, even without this it is impossible, since during their maintenance it is necessary to carry out updates software(ON). Distributing these updates using removable media very inconvenient, and if there are a large number of computers in the serviced local network, it is almost impossible. In this case, the rational option is to organize local repositories of packages previously downloaded from the Internet. The two main approaches to solving this problem using the example of Ubuntu systems will be further described in this article.

How do package repositories work on Linux systems?

Developers provide (SMS) with special links to support their distributions and provide a comfortable user experience. They point to remote servers, which store the most up-to-date and developer-tested software packages for a given distribution. Thanks to these links, the EMS "knows" when and where to download and install package updates. These links can point to both a remote resource and a local one. In the second case, it can be either another computer on the local network, or a local drive and / or even, if you try, an optical drive.

Deb http://ru.archive.ubuntu.com/ubuntu/ bionic universe

This is one of the system repositories included in the distribution initially. There are also repositories organized by individual verified users, for example:

Deb https://repos.codelite.org/ubuntu/ bionic universe

This is a repository created by the developer of the CodeLite development environment, specifically for Ubuntu. And this link has been added to the file sources.list already manually by the user-administrator of the computer. After that it becomes possible automatic installation current and stable versions of CodeLite packages, as well as their updates. And this is how a link to a repository stored on optical media might look like:

Deb cdrom:/ bionic main multiverse restricted universe

As seen, keyword, which defines the access protocol is the value after "deb". For optical media it is "cdrom" and for network access it is "https".
It turns out that the sources of repositories can be supplemented at your own discretion, having previously organized the package repository accordingly.

Using a proxy to organize a local repository

This method involves accessing the repositories through a cache on a proxy computer that has a direct connection to the Internet. The mechanism of operation of such a local repository is as follows:

on a client machine, a package is normally requested to be installed/updated via a server computer;
the requested package is downloaded by the server, stored in a specially designated cache storage and then becomes available to all other clients;
The Apache web server acts as a package distributor for clients, so its installation is required.

So, first you need to install everything you need, i.e. the web server and the package caching utility itself:

$ sudo apt-get install apache2 apt-cacher

When installing apt-cacher a configuration dialog will be displayed, in which you can configure the desired behavior of the utility, for example, set autorun and work in daemon mode. Also, these and some other important settings can be made (for example, using the editor) in configuration file/etc/default/apt-cacher . To enable apt-cacher autorun, set the AUTOSTART parameter to "1":

$ sudo nano /etc/default/apt-cacher . . . # Set to 1 to run apt-cacher as a standalone daemon, set to 0 if you are going # to run apt-cacher from /etc/inetd or in CGI mode (deprecated). alternatively, # invoking "dpkg-reconfigure apt-cacher" should do the work for you. # AUTOSTART=1 . . .

$ sudo nano /etc/apt-cacher/apt-cacher.conf . . . ## Uncomment and set the IP range ## allowed_hosts = 192.168.1.105 - 192.168.1.125 #denied_hosts = . . .

As you can see, the range of required IP addresses is simply indicated. After saving the settings made, you need to restart the Apache web server:

$ sudo service apache2 restart

$ sudo systemctl restart apache2

Now you need to tell clients where they need to go to install packages and updates. To do this, on client machines, you need to create the /etc/apt/apt.conf.d/01proxy file using the same nano editor:

$ sudo nano /etc/apt/apt.conf.d/01proxy

And add the following line to it:

Acquire::http::Proxy "http://192.168.1.100:3142";

Here as the address of the server on which it is installed and running apt-cacher 192.168.1.100 is indicated. Of course, it can be any other address configured for this server.

Now you can check the operation of the local repository (more precisely, remote, but accessible through a proxy) by running the command to update data on available packages:

$ sudo apt-get update

APT-MIRROR - a full-fledged local repository

This method is more "advanced" than using apt cache. Since it implies the presence of a full-fledged package repository directly on the local computer / server or on the local network. But first, such a repository must be created by loading all the necessary packages into it. As in the case with apt-cache, the web server acts as a package distributor. The procedure for setting up a local repository using the apt-mirror utility is as follows:

installation of necessary packages: apt-mirror and apache2;
creation local storage and setting up sources for downloads, downloading packages to the repository;
opening access to ready-made storage for clients;
configuring clients to use the local repository.

So, installing the necessary utilities and packages:

$ sudo apt-get install apache2 apt-mirror

$ sudo mkdir /localrepo

Now in the configuration file /etc/apt/mirror.list you need to edit the line with the "set base_path" instruction. By specifying the newly created directory for the repository in it:

$ sudo nano /etc/apt/mirror.list ############ config ################## # set base_path /localrepo

Further, in the same file, you can add the necessary repositories from which packages will be downloaded. You can copy all standard repositories from /etc/apt/sources.list .
After saving the settings, you can start downloading packages with the command:

$ sudo apt-mirror

This may take a long time, depending on the speed of your Internet connection. It is very useful to add this command to the list of regular procedures so that the local repository is updated automatically.

After the local repository is fully downloaded, its contents should be something like this:

$ ls /localmirror mirror skel var

For the subsequent convenience of setting up clients, it is useful to create on the storage that is contained in the mirror directory:

$ cd /localrepo $ sudo ln -s /localrepo/mirror/us.archive.ubuntu.com/ubuntu/ ubuntu

The ubuntu link will now be used to set the client-side repositories by editing the /etc/apt/sources.list file:
Having opened this file (using the command) with the nano editor, we now need to add the following repositories to it:

Deb http://192.168.1.100/ubuntu trusty universe deb http://192.168.1.100/ubuntu trusty main restricted deb http://192.168.1.100/ubuntu trusty-updates main restricted . . .

Here, 192.168.1.100 is the IP address of the computer where the local repository was created and configured.
Now, to work with packages, you can use the usual apt commands:

$ sudo apt-get update $ sudo apt-get install packagename

Conclusion

In conclusion, it should be recalled that the methods of organizing local repositories described above are suitable for systems based on the debian package format. For RPM-based systems, other tools should be used.

If you find an error, please highlight a piece of text and click Ctrl+Enter.

Instead of using dpkg -i package_name.deb to install, you can create a local repository for packages brought on a flash drive or other media and then use synaptic/kynaptic (or apt-get install package_name).

1. All actions are best done as root

sudo su

2.Now install the dpkg-dev package. It will contain the dpkg-scanpackages utility, which is what we need.

apt-get install dpkg-dev

3.Place all .deb packages in a directory for example /myrepo (and not directly in this directory, but in subdirectories located in this directory)

4. Complete

cd /myrepo dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz

5. Now you need to add a new repository to /etc/apt/sources.list. Insert a line in /etc/apt/sources.list (better to insert closer to the beginning, because apt-get (synaptic/kynaptic) prioritizes the source that is earlier than the rest)

deb file:/myrepo ./

6. Update the list of repositories

apt-get update

If the contents of the package directory change, repeat steps 1.4.

21.07.2020

Internet

The most interesting: