0.17: Release Notes

v0.17 was released on March 17. Several important features were implemented in Debian-adapter:

  • GPG clearsign signature is now supported by the adapter, meaning that Release index files are now signed with gpg;
  • Duplicated packages handling: while updating Packages index we now check that the package to add is not present in the repository yet. If it does, the old package and metadata are removed and replaced with the new one;
  • Debian-repository now provides some sdk and can be used as a dependency for the debian repository batch-update and indexes generation. For more details check Debian interface and its Debian.Asto implementation.

We started to restructure Artipie integration tests, changes are mostly infrastructural: now there are two Docker containers started on each integration test, Artipie docker image built on package state and client image. Client image differs depending on the repository we are testing, it could be maven:3.6.3-jdk-11 for maven repository tests or centos:centos8 to verify rpm repository. Thus, our tests work with the image we provide for our users guaranteeing its efficiency and credibility.

Here is the list of other implemented features and important fixes:

0.16: Release Notes

v0.16 was released on February 16. Here is the list of other implemented features and important fixes:

  • Docker-adapter now supports overwrite permission for tags
  • Release index file is now fully supported in Debian-adapter
  • FileStorage delete operation was improved: now we also remove empty directories recursively along with the removed file. This change fixed connected bug in Docker-adapter
  • Basic authentication was added to Composer-adapter
  • Writing big files operation performance was improved in Asto
  • Npm-adapter now supports tgz upload, to publish the package simple curl call can be used: curl PUT your-archive.tgz. Also, unpublish command now can remove the specified version of the package, not only the package entirely.

Granular Permissions For Docker Registry

Docker-adapter supports granular permissions, which means that some operations can be granted for specific scope and image. Here is the example docker repository settings yaml:

repo:
  type: docker
  storage:
    type: fs
    path: /var/artipie/data
  permissions:
    alice:
      - repository:my-alpine:*
      - repository:ubuntu-latest:pull
      - repository:some-image:overwrite
      - repository:some-image:push
    mike:
      - repository:some-image:push
      - repository:some-image:pull
    bob:
      - registry:base:*
      - registry:catalog:*

Each permission consists of three parts: scope:name:action, scope can be either “registry” for the registry level operations or “repository”. Name is the access resource name, action can be one of pull, push, overwrite or * (* stands for any action), overwrite works for overwriting existing tags and allows creating new tags by default.

0.15: Release Notes

v0.15 was released on January 29. We are happy to announce that minimal support for Debian repository was added to Artipie: you can configure the repository which apt-get will be able to work with.

Npm-adapter functionality was extended with the following commands support:

Docker-adapter evolution:

  • In practice, it seems that blobs are always uploaded in single chunk, that’s why we optimized this particular upload case
  • Image level permissions are now supported, meaning, that you can define which user has read or write access to particular image

And here is the list of other implemented features:

  • Helm-adapter now supports deleteByName and deleteByNameAndVersion commands and --merge flag for indexing
  • Php-adapter became fully non-blocking
  • Nuget-adapter endpoints were refactored not to interact with data storage directly

0.14: Release Notes

v0.14 was released on December 30. This month our two main directions were Docker-adapter and management-api transfer. Features added to Docker-adapter:

Artipie API and dashboard were transferred to management-api repository and added minimal integration tests for API endpoints in Artipie.

Here is the list of other changes:

We also started to develop Debian-adapter to support Debian repository.

0.13: Release Notes

v0.13 was released on November 27. Here is the list of the key changes:

Python And Binary Repositories

Being a student I dreamed about something like Artipie, I needed some private centralized storage where I could have kept both binary or text data files and Python packages. We were conducting a research and used to work with a lot of data from experimental measurements and used Python to process and visualise them. My university colleagues from that time were mathematicians, they did not have strong development background and were absolutely fine with passing data and code via flash drive or email. All my efforts to inculcate VCS (GitHub, for example, provides free Pro accounts for students and professors) and some cloud storage usage were always postponed to the brighter future. I’ve quit the postgraduate program and never saw that future to come.

Now, I’d like to show how I could have used Artipie being that student. In the example I’ll use Artipie Central, but the same will work with an Artipie instance running on your private server in the private network.

Python Repository

Artipie supports PyPi repository which twine and pip can perfectly understand. This means that you can work with Artipie Python repository exactly as you do while installing or publishing packages from or to PyPI and TestPyPI repos. Here is my PyPi repository configuration from Artipie Central:

repo:
  type: pypi
  storage: default
  permissions:
    olenagerasimova:
      - download
      - upload
    "*":
      - download

Upload is available for me only, but anyone can download any package, more details about permission configuration can be found here.

To make sure this repository exists and works, open index page from your browser, packages list should be displayed there.

I’ve created a small Python example project, source code is available here, the main idea of the example is from my university times: download three data files from Artipie Central (the next part is about how these files got there), read the numbers into arrays and use these arrays to draw a plot. Let’s use pip to install this example package and run it:

python3 -m pip install --index-url https://central.artipie.com/olenagerasimova/pypi/ pypiexample
python3 -m pypiexample

If you run these commands then you will see a polar plot with three curves, visualisation of our data files.

To publish package to the Artipie Central repository build it with sdist and use twine for upload:

python setup.py sdist bdist_wheel
twine upload --repository-url https://central.artipie.com/olenagerasimova/pypi -u olenagerasimova -p *** dist/*

I’ve set up GitHub action to upload the example to Artipie Central on release, check the script here.

To sum up, in this chapter we set up PyPi repository in Artipie Central, created sample Python project, published and installed it. Now we’ll move on to the binary storage and I’ll show where data files for the example are stored.

Binary Storage

Artipie can be used as a storage for any files, this repository is used to store experimental data which are visualised by the Python example. I’ve created binary data repository on Central, exact configuration is the following:

repo:
  type: file
  storage: default
  permissions:
    olenagerasimova:
      - upload
      - download
    "*":
      - download

As in the previous example I’ve allowed download operation to anyone. Data files are three text files with some numbers, here is how the first one looks like:

6
3.5
5
4
4.5
3
2.7
5
6
3
1.2
3.2
6

The other two files have the same form, only numbers are different. These files can be uploaded to the Artipie Central data repository with simple PUT http request:

httpie -a olenagerasimova:*** PUT https://central.artipie.com/olenagerasimova/data/y1.dat @data/y1.dat
httpie -a olenagerasimova:*** PUT https://central.artipie.com/olenagerasimova/data/y2.dat @data/y2.dat
httpie -a olenagerasimova:*** PUT https://central.artipie.com/olenagerasimova/data/y3.dat @data/y3.dat

To download the files use GET request

httpie -a https://central.artipie.com/olenagerasimova/data/y1.dat > ./data/y1.dat

or simply open links one, two and three in the browser. The Python example downloads these files and visualise them on the polar plot.

As this binary repository API is very simple (http PUT and GET requests), it’s easy to write a piece of code in any language to upload and downloads necessary files. What is also important, authorization is supported in Artipie, so it’s possible to grant rights to upload/download from the repository how it suits your case.

Using Artipie as a Maven repository

As we know, Apache Maven is one of the most popular build management tool for Java. Many companies that do Java development have private Maven repositories, where they store all the artifacts. There are some problems with management when many developers work with a huge repository.
Let’s assume, you need to restrict access to artifacts. So, it’s necessary to split this one into several small repositories or add authentication for users. But at the same time you want to keep the ability to share easily and quickly within a team or between multiple teams of developers. Or you are just wondering how to set up an private repository for a pet project.
Artipie allows you to solve these problems as it is a binary artifact management tool. In the blog post I will share with you how to configure Artipie and project for this purpose. Let’s start!

In the simplest case, you need a file with the contents below. We want to use file storage therefore we specify fs as the type. At first, create a separate directory with the following artipie.yaml file inside:

meta:
  storage:
    type: fs
    path: /var/artipie/configs

After that you can install and run Artipie with Docker:

$ docker run -d --name artipie \
             -v $(pwd)/artipie.yaml:/etc/artipie.yml \
             -v $(pwd):/var/artipie \
             -p 8080:80 \
             artipie/artipie:latest

In the previous step, we created the settings file for Artipie. Now let’s see what the configuration file for the repository should look like. For this, we need to create configs directory and the following my-maven.yaml file inside:

repo:
  type: maven
  storage:
    type: fs
    path: /var/artipie/data

As a results, you will have the following files structure:

.
├── artipie.yaml
├── configs
    └── my-maven.yaml

At this step, I will show you how to configure your project to use created Artipie Maven repository. If you want to mvn deploy your project to Artipie, add the following to project’s pom.xml:

<distributionManagement>
    <repository>
        <id>artipie</id>
        <url>http://localhost:8080/my-maven</url>
    </repository>
</distributionManagement>

In case you want to mvn install deployed packages from Artipie then add the following to project’s pom.xml:

<repositories>
    <repository>
        <id>artipie</id>
        <url>http://localhost:8080/my-maven</url>
    </repository>
</repositories>

If you have completed all the instructions correctly now you should be able to use the repository: deploy and install Maven artifacts.

Configure Artipie with authentication

If you want to use authentication for users then you need to add a section with credentials to the artipie.yaml. In this case, the file with settings should look like this:

meta:
  storage:
    type: fs
    path: /var/artipie/configs
  credentials:
    type: file
    path: _credentials.yml

Also, it’s necessary to create _credentials.yml file in the configs directory. Each node is a user name. The supported types of password can be found here The file with credentials should look like this:

credentials:
  alice:
    type: plain
    pass: 123
  bob:
    type: plain
    pass: qwerty

As a results, you will have the following files structure:

.
├── artipie.yaml
├── configs
    ├── my-maven.yaml
    └── _credentials.yml

Don’t forget to run Artipie server with Docker and to update pom.xml of your project as shown above. If all the steps were done correctly you should be able to use the repository with authentication: deploy and install Maven artifacts.

Apart from demonstrated possibilities, Artipie Maven supports: group repositories, proxies and user groups. Check out the corresponding documentation section for more details.

In this blog post I explained how to set up private Maven repository with Artipie and how to configure a Maven project to use it. I hope now it will be easier and less time-consuming to manage the repository by using Artipie.

0.12: Release Notes

v0.12 was released on October 29. In October, we focused on extending and adding proxy for our adapters. First, we implemented basic and bearer authentication in our http-client, then supported authentication for proxy in different adapters. Here is the list of the proxy adapters, authentication and caching in the local storage can be configured for each proxy:

We also worked on another features in Artipie: