Known Issues

The user namespace (userns) feature of the Linux kernel adds another security layer to Linux containers. It allows a host machine to run containers outside its user ID (UID) or group ID (GID) namespace. This means that all containers can have a root account (UID 0) in their own namespace, and run processes without receiving root privileges from the host machine. When a userns is created, the Linux kernel provides a mapping between the container and the host machine. For example, if you start a container and run a process with UID 0 inside it (usually root account in the container), the Linux kernel maps the container's UID 0 to a non-privileged UID on the host machine. This allows the container to run a process as if it were the root user, while actually being run by the non-root user on the host machine.

The error is caused by a userns remapping failure. When the host machine files are configured with a valid UID or GID for remapping, the UID or GID must be in the range of 0–65535. When a job starts the Docker container, Docker pulls an image and extracts layers from that image. If a layer contains files with UID or GID outside of the accepted range, then Docker can't successfully remap so the container fails to start.

For example, if a file exists with UID and GID higher at the allowed range. If you copy this file into the Docker image, it can keep the high UID and GID.

If you run the container image and the file must be used in the container, then the container fails.

Ensure that none of the files that are used in your Docker image have high UID or GID.

If you don't know which file has high UID or GID in your image, you can find them with:

1. Enter the container with:

   docker run it <image-name> sh

2. Search for files with high UID/GID:

   • Find users:

     find / \( -uid 1000000 \) -ls 2>/dev/null

   • Find groups:

     find / \( -gid 1000000 \) -ls 2>/dev/null

   The 1000000 number is different because it is the ID error.

3. If you find files, ensure that the UID or GID is lower, either in the location where the file is stored or in the container.

In your Docker file, after you copy the files you need in the image run one of the following:

On the folder:

RUN chown -R root:root /root 

Directly on the file:

RUN chown -R root:root job_logs.py

This error occurs when you use a local file system storage that is not in the /home/datascience location.

In jobs, you can specify the size of the block storage. The block storage is mounted to the local folder /home/datascience that you can use during the job run. It has a storage size equivalent to the size set of the block storage before the job run. If you use or create a directory outside of this location, it can run quickly out of space and the error message appears. It depends on the size of the content being stored.

Ensure that you always work under the main folder that has the full size of the block storage, which is /home/datascience. Create, edit, and download all content in this location. Create directories under this location.

By default, Docker on an M1 MacBook creates linux/arm64 images, which only work on the machines that are using ARM architecture. Intel-based machines use AMD architecture. As a result, docker images built on an M1 MacBook might not work on Intel-based machines.