rogs/yams
rogs/yams
1
0
Fork 0
Code Issues 9 Pull Requests Actions Packages Projects Releases Wiki Activity

Add hardlinking support to yams. #67

Closed
sGfU wants to merge 1 commits from master into master
pull from: master
merge into: rogs:master
Conversation 13 Commits 1 Files Changed 1 +11 -11
sGfU commented 2024-04-03 02:42:52 -03:00 (Migrated from gitlab.com)
Copy Link

Tested by UncarvedMatt in the discord, unverified by myself so please test before adding Rodger!

Tested by UncarvedMatt in the discord, unverified by myself so please test before adding Rodger!
rogs commented 2024-04-03 08:52:05 -03:00 (Migrated from gitlab.com)
Copy Link

This should work! I'll test it this weekend before merging. Thanks!

This should work! I'll test it this weekend before merging. Thanks!
fryfrog commented 2024-04-05 20:32:54 -03:00 (Migrated from gitlab.com)
Copy Link

I have some suggestions:

  • Use a named download folder for each type of download, like usenet/ and torrents/. That way users can use either or both w/o having folder conflicts or putting both in the same place.
  • Use a library folder for all the library folders, like /data/library/{TV|Movies}.
  • Pass into containers only what they need. For example:
    • sonarr/radarr need everything so they get something like /outside_folder_data:/data
    • torrent client only needs torrent folder so something like /outside_folder_data/torrents:/data/torrents
    • usenet client only needs usenet folder so something like /outside_folder_data/usenet:/data/usenet.
    • Plex and bazarr only need the library, so like /outside_folder_data/library:/data/library.
  • Pass in UMASK=022 just in case. Double check each image uses that, I've seen some that use UMASK_SET=.
I have some suggestions: * Use a named download folder for each type of download, like `usenet/` and `torrents/`. That way users can use either or both w/o having folder conflicts or putting both in the same place. * Use a library folder for all the library folders, like `/data/library/{TV|Movies}`. * Pass into containers *only* what they need. For example: * sonarr/radarr need everything so they get something like `/outside_folder_data:/data` * torrent client only needs torrent folder so something like `/outside_folder_data/torrents:/data/torrents` * usenet client only needs usenet folder so something like `/outside_folder_data/usenet:/data/usenet`. * Plex and bazarr only need the library, so like `/outside_folder_data/library:/data/library`. * Pass in `UMASK=022` just in case. Double check each image uses that, I've seen some that use `UMASK_SET=`.
fryfrog commented 2024-04-05 20:33:35 -03:00 (Migrated from gitlab.com)
Copy Link

I also wouldn't even show bad paths in #comments, they'll just be confusing.

I also wouldn't even show bad paths in #comments, they'll just be confusing.
patrickdundas commented 2024-06-18 23:22:41 -03:00 (Migrated from gitlab.com)
Copy Link

Any movement on this?

Any movement on this?
rogs commented 2024-06-19 08:32:03 -03:00 (Migrated from gitlab.com)
Copy Link

Sadly, no update yet 😞 I need to test this and update the documentation before we can merge it. You can also use it manually if you run YAMS from sGfU/yams:master

Sadly, no update yet :disappointed: I need to test this and update the documentation before we can merge it. You can also use it manually if you run YAMS from [sGfU/yams:master](https://gitlab.com/sGfU/yams/-/tree/master)
militaryk commented 2024-08-24 07:35:15 -03:00 (Migrated from gitlab.com)
Copy Link

Just wondering if theirs been any updates on this. Been trying to follow, Trash Guides tutorials but am having issues left right, and center.

Just wondering if theirs been any updates on this. Been trying to follow, Trash Guides tutorials but am having issues left right, and center.
rogs commented 2024-09-24 10:07:21 -03:00 (Migrated from gitlab.com)
Copy Link

I'm rewriting the documentation to reflect the new changes. You can safely use this PR as base, but you'll need to configure everything manually.

I'm rewriting the documentation to reflect the new changes. You can safely use this PR as base, but you'll need to configure everything manually.
MasonStooksbury commented 2024-10-08 21:06:35 -03:00 (Migrated from gitlab.com)
Copy Link

Assuming I'm reading this correctly, if someone installed YAMS on Machine A, but all their media lived in an external drive, NAS, etc; would this still work?

TRaSH guides explicitly call out that hardlinks can't happen across "systems, partitions, volumes, and mounts" (https://trash-guides.info/Hardlinks/Hardlinks-and-Instant-Moves/#hardlinks-limitations)

There's quite a few people with this "split setup". Not sure how we could accommodate both, but one option could be modifying the install script to construct the docker-compose.yaml based on whether or not the user selected hardlinking

In that case, I think the default should be non-hardlinked (since it's the simplest and works for most people), but then give the option to select a "hardlinked" install

Thoughts?

Assuming I'm reading this correctly, if someone installed YAMS on Machine A, but all their media lived in an external drive, NAS, etc; would this still work? TRaSH guides explicitly call out that hardlinks can't happen across "systems, partitions, volumes, and mounts" (https://trash-guides.info/Hardlinks/Hardlinks-and-Instant-Moves/#hardlinks-limitations) There's quite a few people with this "split setup". Not sure how we could accommodate both, but one option could be modifying the install script to construct the docker-compose.yaml based on whether or not the user selected hardlinking In that case, I think the default should be non-hardlinked (since it's the simplest and works for most people), but then give the option to select a "hardlinked" install Thoughts?
rogs commented 2024-10-09 08:23:28 -03:00 (Migrated from gitlab.com)
Copy Link

It does work; I have it running perfectly on my server that's connected to a NAS. You just need to ensure that everything is mounted on the same volume.

TRaSH mentions that the issue arises across separate file systems, partitions, volumes, or mounts—not if it's the same file system, partition, volume, or mount 😉

It does work; I have it running perfectly on my server that's connected to a NAS. You just need to ensure that everything is mounted on the same volume. TRaSH mentions that the issue arises **across separate** file systems, partitions, volumes, or mounts—not if it's the **same file** system, partition, volume, or mount :wink:
MasonStooksbury commented 2024-10-17 14:20:27 -03:00 (Migrated from gitlab.com)
Copy Link

I might just be really dumb, but I think I'm missing something

Your setup has your YAMS stack and your media on the same NAS? If so, I can see how this would work.

But if I have YAMS installed on a different disk, and my media is stored on an external harddrive, wouldn't that be two different file systems/mounts? Meaning there would be issues?

I might just be really dumb, but I think I'm missing something Your setup has your YAMS stack and your media on the same NAS? If so, I can see how this would work. But if I have YAMS installed on a different disk, and my media is stored on an external harddrive, wouldn't that be two different file systems/mounts? Meaning there would be issues?
rogs commented 2024-10-17 14:29:42 -03:00 (Migrated from gitlab.com)
Copy Link

Let's assume:

  • YAMS is running in computer A.
  • Your NAS is running in computer B.
  • Your media lives in the NAS in a SMB share called "Media".
  • The YAMS VM has the SMB share mounted somewhere in the filesystem. For the sake of argument, let's say it's mounted in /mnt/media.
  • Both your computer and NAS are different machines connected via LAN.

If you set your MEDIA_DIRECTORY as /mnt/media, hardlinking should work without any issues. The problem (what TRaSH describes) is when you mount multiple volumes in docker and try hardlinking between them. You can see the changes I've made for V3 here: https://gitlab.com/rogs/yams/-/blob/v3/docker-compose.example.yaml?ref_type=heads#L13. Basically, now all media gets mounted in a single volume, allowing hardlinking by default.

Let's assume: - YAMS is running in computer A. - Your NAS is running in computer B. - Your media lives in the NAS in a SMB share called "Media". - The YAMS VM has the SMB share mounted somewhere in the filesystem. For the sake of argument, let's say it's mounted in `/mnt/media`. - Both your computer and NAS are different machines connected via LAN. If you set your `MEDIA_DIRECTORY` as `/mnt/media`, hardlinking should work without any issues. The problem (what TRaSH describes) is when you mount **multiple volumes** in docker and try hardlinking between them. You can see the changes I've made for V3 here: https://gitlab.com/rogs/yams/-/blob/v3/docker-compose.example.yaml?ref_type=heads#L13. Basically, now all media gets mounted in a single volume, allowing hardlinking by default.
MasonStooksbury commented 2024-10-17 17:05:38 -03:00 (Migrated from gitlab.com)
Copy Link

OHH. I think it's clicked now. That makes a lot more sense. Thanks for the explanation!

OHH. I think it's clicked now. That makes a lot more sense. Thanks for the explanation!
rogs commented 2024-12-26 12:04:15 -03:00 (Migrated from gitlab.com)
Copy Link

Closed in favor of !22

Closed in favor of !22
rogs (Migrated from gitlab.com) closed this pull request 2024-12-26 12:04:16 -03:00

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
add-to-docs
bug
done
feature-request
in progress
troubleshooting
v2
yams extra
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
rogs
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: rogs/yams#67
No description provided.
Delete Branch "master"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?