Current method of mounting docker volumes does not allow for hardlinking #30

New Issue

2024-01-09T22:13:51-03:00

k-saw commented

2024-01-09 22:13:51 -03:00

(Migrated from gitlab.com)

The current example docker-compose has the containers for Sonarr, Radarr, etc. mounting the folders for media into different volumes than the downloads.

    volumes:
      - ${MEDIA_DIRECTORY}/tvshows:/tv
      - ${MEDIA_DIRECTORY}/downloads:/downloads
      - ${INSTALL_DIRECTORY}/config/sonarr:/config

When a download finishes and *Arr imports it, it will attempt to hardlink and fail (because you can't hardlink across volumes). It then falls back to doing a copy. This is really slow (relatively speaking), puts additional wear on the drive, and doubles the storage space needed.

The *arrs should mount the entire MEDIA_DIRECTORY as a single volume and the relevant torrent program should mount to the same container path.
I've manually edited my docker-compose to the following setup but I imagine any yams update will wipe this out.

  qbittorrent:
    image: lscr.io/linuxserver/qbittorrent
    container_name: qbittorrent
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - WEBUI_PORT=8080
    volumes:
      - ${MEDIA_DIRECTORY}/downloads:/data/downloads
      - ${INSTALL_LOCATION}/config/qbittorrent:/config

  sonarr:
    image: lscr.io/linuxserver/sonarr
    container_name: sonarr
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
    volumes:
      - ${MEDIA_DIRECTORY}:/data
      - ${INSTALL_LOCATION}/config/sonarr:/config

  radarr:
    image: lscr.io/linuxserver/radarr
    container_name: radarr
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
    volumes:
      - ${MEDIA_DIRECTORY}:/data
      - ${INSTALL_LOCATION}/config/radarr:/config

And so on.

The root paths in the *arrs would have to be updated as well as the save path in qBittorrent or whatever.

Slightly adapted from https://trash-guides.info/Hardlinks/How-to-setup-for/Docker/

The current example `docker-compose` has the containers for Sonarr, Radarr, etc. mounting the folders for media into different volumes than the downloads. ```yaml volumes: - ${MEDIA_DIRECTORY}/tvshows:/tv - ${MEDIA_DIRECTORY}/downloads:/downloads - ${INSTALL_DIRECTORY}/config/sonarr:/config ``` When a download finishes and \*Arr imports it, it will attempt to hardlink and fail (because you can't hardlink across volumes). It then falls back to doing a copy. This is really slow (relatively speaking), puts additional wear on the drive, and doubles the storage space needed. The *arrs should mount the entire `MEDIA_DIRECTORY` as a single volume and the relevant torrent program should mount to the same container path. I've manually edited my docker-compose to the following setup but I imagine any `yams update` will wipe this out. ```yaml qbittorrent: image: lscr.io/linuxserver/qbittorrent container_name: qbittorrent environment: - PUID=${PUID} - PGID=${PGID} - WEBUI_PORT=8080 volumes: - ${MEDIA_DIRECTORY}/downloads:/data/downloads - ${INSTALL_LOCATION}/config/qbittorrent:/config sonarr: image: lscr.io/linuxserver/sonarr container_name: sonarr environment: - PUID=${PUID} - PGID=${PGID} volumes: - ${MEDIA_DIRECTORY}:/data - ${INSTALL_LOCATION}/config/sonarr:/config radarr: image: lscr.io/linuxserver/radarr container_name: radarr environment: - PUID=${PUID} - PGID=${PGID} volumes: - ${MEDIA_DIRECTORY}:/data - ${INSTALL_LOCATION}/config/radarr:/config ``` And so on. The root paths in the *arrs would have to be updated as well as the save path in qBittorrent or whatever. Slightly adapted from https://trash-guides.info/Hardlinks/How-to-setup-for/Docker/

hobsie commented

2024-01-10 06:29:24 -03:00

(Migrated from gitlab.com)

Hit this exact same thing yesterday and made precisely the same changes you have. Went from taking nearly an hour to copy a 40GB file on my setup from /Downloads to /Movies (despite being on the same physical drive) to being instant.

Would be good to see this as the default setup

Hit this exact same thing yesterday and made precisely the same changes you have. Went from taking nearly an hour to copy a 40GB file on my setup from /Downloads to /Movies (despite being on the same physical drive) to being instant. Would be good to see this as the default setup

rogs commented

2024-01-10 08:10:50 -03:00

(Migrated from gitlab.com)

Hey!

You are absolutely 100% right. This is a change I wanted to do for a while, but there are 2 things that bother me now:

It will break existing setups if people run yams update (easily fixable by changing the docker-compose filename)
The documentation change is going to be huge, and I don't have time to work on docs right now (maybe in a couple of weeks)

We have discussed some of these changes in the Discord server, so if you want to join the conversation that would be greatly appreciated!

This is definitely top priority right now 👍

Hey! You are absolutely 100% right. This is a change I wanted to do for a while, but there are 2 things that bother me now: 1. It will break existing setups if people run `yams update` (easily fixable by changing the docker-compose filename) 2. The documentation change is going to be huge, and I don't have time to work on docs right now (maybe in a couple of weeks) We have discussed some of these changes in [the Discord server](https://discord.com/invite/Gwae3tNMST), so if you want to join the conversation that would be greatly appreciated! This is definitely top priority right now :thumbsup:

rogs commented

2024-01-10 08:10:53 -03:00

(Migrated from gitlab.com)

assigned to @rogs

methbkts commented

2024-01-12 11:16:43 -03:00

(Migrated from gitlab.com)

For the 1. we can make a backup of the docker-compose file (cp docker-compose.yaml docker-compose.yaml.bak) before updating docker-compose.example.yaml

For the 1. we can make a backup of the docker-compose file (`cp docker-compose.yaml docker-compose.yaml.bak`) before updating `docker-compose.example.yaml`

rogs commented

2024-01-12 11:17:58 -03:00

(Migrated from gitlab.com)

This is a good idea for new users, but when old users upgrade they are going to get overwritten 😕

This is a good idea for new users, but when old users upgrade they are going to get overwritten :confused:

rogs commented

2024-12-23 12:32:21 -03:00

(Migrated from gitlab.com)

Fixed on V3. Will close this issue when v3 is out.

rogs commented

2024-12-26 12:10:32 -03:00

(Migrated from gitlab.com)