rogs/yams
rogs/yams
1
0
Fork 0
Code Issues 9 Pull Requests Actions Packages Projects Releases Wiki Activity

Start YAMS in stages for Testing & Low Power servers #31

New Issue
Closed
opened 2024-03-01 12:49:43 -03:00 by cooooosh · 13 comments
cooooosh commented 2024-03-01 12:49:43 -03:00 (Migrated from gitlab.com)
Copy Link

Feature Request

Hoping to create additional YAMS CLI startup commands to start docker-compose.yaml and docker-compose.custom.yaml separately. This would help in two cases:

  1. Testing/adding/updating containers in the custom compose file without restarting core services (bittorrent, radarr, sonarr, etc.)
  2. Users with low power/underpowered hosts that have trouble with starting up ~20+ docker containers at once. I'm running this on an old Dell Thin Client with a J5005. It can handle most everything I throw at it once it's up and running, but the initial startup can take several minutes and sometimes errors out.

I get that this may be more confusing for some and might not make sense to include for all installs, but hoping this could at least become an option during install. I took a pass at an example on my own, and created a templatized version for install below. If there's interest I can take a pass at a modified install.sh as well!

/usr/local/bin/yams.v2 
#!/bin/bash
set -euo pipefail


dc="docker compose -f <filename> -f <custom_file_filename>"
dca="docker compose -f <filename>"
dcb="docker compose -f <custom_file_filename>"
install_directory="<install_directory>"

option=${1:-"--help"}

help() {
   echo "yams - Yet Another Media Server"
   echo
   echo "Usage: yams [--help|restart|stop|stop-custom|start|start-base|start-custom|destroy|check-vpn|update]"
   echo "options:"
   echo "--help         displays this help message"
   echo "restart        restarts yams services"
   echo "stop           stops all yams services"
   echo "stop-custom    stops custom yams services"
   echo "start          starts all yams services"
   echo "start-base     starts basic yams services"
   echo "start-custom   starts custom yams services"
   echo "destroy        destroy yams services so you can start from scratch"
   echo "check-vpn      checks if the VPN is working as expected"
   echo "update         updates YAMS"
}

send_success_message() {
    echo -e "$(printf "\e[32m$1\e[0m")"
}

send_error_message() {
    echo -e "$(printf "\e[31m$1\e[0m")"
    exit 255
}

if [ "$option" == "--help" ]; then
    help
    exit 0
fi

if [ "$option" == "restart" ]; then
    $dc stop && $dc up -d
    echo "YAMS is starting. Wait 1 min until all the services are up and running..."
    exit 0
fi

if [ "$option" == "stop" ]; then
    $dc stop
    exit 0
fi

if [ "$option" == "stop-custom" ]; then
    $dcb stop
    echo "YAMS has stopped custom services. Base services are still running..."
    exit 0
fi

if [ "$option" == "start" ]; then
    $dc up -d
    echo "YAMS is starting. Wait 1 min until all the services are up and running..."
    exit 0
fi

if [ "$option" == "start-base" ]; then
    $dca up -d
    echo "YAMS is starting base services. Wait 1 min until all the services are up and running..."
    exit 0
fi

if [ "$option" == "start-custom" ]; then
    $dcb up -d
    echo "YAMS is starting custom services. Wait 1 min until all the services are up and running..."
    exit 0
fi

if [ "$option" == "check-vpn" ]; then
    echo "Getting your IP..."
    your_ip=$(curl -s api.ipify.org)
    echo "$your_ip"
    echo "Your local IP country is $(curl -s https://am.i.mullvad.net/country)"
    echo
    echo
    echo "Getting your qBittorrent IP..."

    qbittorrent_ip=$(docker exec qbittorrent sh -c "curl -s api.ipify.org");
    if [ -n "$qbittorrent_ip" ]; then
        echo "$qbittorrent_ip"
        echo "Your country in qBittorrent is $(docker exec -it qbittorrent sh -c 'curl -s https://am.i.mullvad.net/country')"
        if [ "$qbittorrent_ip" == "$your_ip" ]; then
            send_error_message "Your IPs are the same! qBittorrent is exposing your IP! ⚠️"
        else
            send_success_message "Your IPs are different. qBittorrent is masking your IP! ✅ "
        fi
    else
        send_error_message "Failed to retrieve qBittorrent IP. Please check your setup. ⚠️"
    fi
fi

if [ "$option" == "destroy" ]; then
    echo
    echo
    read -p "Are you sure you want to destroy all your yams services? THIS IS NOT RECOVERABLE! ⚠️ ️🚨 [y/N]: " destroy_now
    destroy_now=${destroy_now:-"n"}
    if [ "$destroy_now" == "y" ]; then
        $dc down
        echo
        echo
        echo "yams services were destroyed. To restart, run: "
        echo "\$ yams start"
    fi
fi

if [ "$option" == "update" ]; then
    echo "Updating YAMS..."
    $dc stop
    rm -rf /tmp/yams && mkdir /tmp/yams
    wget https://gitlab.com/rogs/yams/-/raw/master/docker-compose.example.yaml -O /tmp/yams/docker-compose.example.yml > /dev/null 2>&1
    source $install_directory/.env

    filename="$install_directory/docker-compose.yaml"

    cp /tmp/yams/docker-compose.example.yml $filename


    sed -i -e "s;<media_service>;$MEDIA_SERVICE;g" "$filename"
    if [ "$MEDIA_SERVICE" == "plex" ]; then
        sed -i -e "s|#network_mode: host # plex|network_mode: host # plex|g" "$filename" \
            -e "s|ports: # plex|#ports: # plex|g" $filename \
            -e "s|- 8096:8096 # plex|#- 8096:8096 # plex|g" $filename
    fi

    if [ "$VPN_ENABLED" == "y" ]; then
        sed -i -e "s;#network_mode: \"service:gluetun\";network_mode: \"service:gluetun\";g" "$filename" \
            -e "s;ports: # qbittorrent;#port: # qbittorrent;g" "$filename" \
            -e "s;- 8080:8080 # qbittorrent;#- 8080:8080 # qbittorrent;g" "$filename" \
            -e "s;#- 8080:8080/tcp # gluetun;- 8080:8080/tcp # gluetun;g" "$filename"
    fi

    $dc up -d
    echo "YAMS was updated and it is starting. Wait 1 min until all the services are up and running..."
fi
Feature Request Hoping to create additional YAMS CLI startup commands to start docker-compose.yaml and docker-compose.custom.yaml separately. This would help in two cases: 1. Testing/adding/updating containers in the custom compose file without restarting core services (bittorrent, radarr, sonarr, etc.) 2. Users with low power/underpowered hosts that have trouble with starting up \~20+ docker containers at once. I'm running this on an old Dell Thin Client with a J5005. It can handle most everything I throw at it once it's up and running, but the initial startup can take several minutes and sometimes errors out. I get that this may be more confusing for some and might not make sense to include for all installs, but hoping this could at least become an option during install. I took a pass at an example on my own, and created a templatized version for install below. If there's interest I can take a pass at a modified install.sh as well! <details><summary>/usr/local/bin/yams.v2</summary> ``` #!/bin/bash set -euo pipefail dc="docker compose -f <filename> -f <custom_file_filename>" dca="docker compose -f <filename>" dcb="docker compose -f <custom_file_filename>" install_directory="<install_directory>" option=${1:-"--help"} help() { echo "yams - Yet Another Media Server" echo echo "Usage: yams [--help|restart|stop|stop-custom|start|start-base|start-custom|destroy|check-vpn|update]" echo "options:" echo "--help displays this help message" echo "restart restarts yams services" echo "stop stops all yams services" echo "stop-custom stops custom yams services" echo "start starts all yams services" echo "start-base starts basic yams services" echo "start-custom starts custom yams services" echo "destroy destroy yams services so you can start from scratch" echo "check-vpn checks if the VPN is working as expected" echo "update updates YAMS" } send_success_message() { echo -e "$(printf "\e[32m$1\e[0m")" } send_error_message() { echo -e "$(printf "\e[31m$1\e[0m")" exit 255 } if [ "$option" == "--help" ]; then help exit 0 fi if [ "$option" == "restart" ]; then $dc stop && $dc up -d echo "YAMS is starting. Wait 1 min until all the services are up and running..." exit 0 fi if [ "$option" == "stop" ]; then $dc stop exit 0 fi if [ "$option" == "stop-custom" ]; then $dcb stop echo "YAMS has stopped custom services. Base services are still running..." exit 0 fi if [ "$option" == "start" ]; then $dc up -d echo "YAMS is starting. Wait 1 min until all the services are up and running..." exit 0 fi if [ "$option" == "start-base" ]; then $dca up -d echo "YAMS is starting base services. Wait 1 min until all the services are up and running..." exit 0 fi if [ "$option" == "start-custom" ]; then $dcb up -d echo "YAMS is starting custom services. Wait 1 min until all the services are up and running..." exit 0 fi if [ "$option" == "check-vpn" ]; then echo "Getting your IP..." your_ip=$(curl -s api.ipify.org) echo "$your_ip" echo "Your local IP country is $(curl -s https://am.i.mullvad.net/country)" echo echo echo "Getting your qBittorrent IP..." qbittorrent_ip=$(docker exec qbittorrent sh -c "curl -s api.ipify.org"); if [ -n "$qbittorrent_ip" ]; then echo "$qbittorrent_ip" echo "Your country in qBittorrent is $(docker exec -it qbittorrent sh -c 'curl -s https://am.i.mullvad.net/country')" if [ "$qbittorrent_ip" == "$your_ip" ]; then send_error_message "Your IPs are the same! qBittorrent is exposing your IP! ⚠️" else send_success_message "Your IPs are different. qBittorrent is masking your IP! ✅ " fi else send_error_message "Failed to retrieve qBittorrent IP. Please check your setup. ⚠️" fi fi if [ "$option" == "destroy" ]; then echo echo read -p "Are you sure you want to destroy all your yams services? THIS IS NOT RECOVERABLE! ⚠️ ️🚨 [y/N]: " destroy_now destroy_now=${destroy_now:-"n"} if [ "$destroy_now" == "y" ]; then $dc down echo echo echo "yams services were destroyed. To restart, run: " echo "\$ yams start" fi fi if [ "$option" == "update" ]; then echo "Updating YAMS..." $dc stop rm -rf /tmp/yams && mkdir /tmp/yams wget https://gitlab.com/rogs/yams/-/raw/master/docker-compose.example.yaml -O /tmp/yams/docker-compose.example.yml > /dev/null 2>&1 source $install_directory/.env filename="$install_directory/docker-compose.yaml" cp /tmp/yams/docker-compose.example.yml $filename sed -i -e "s;<media_service>;$MEDIA_SERVICE;g" "$filename" if [ "$MEDIA_SERVICE" == "plex" ]; then sed -i -e "s|#network_mode: host # plex|network_mode: host # plex|g" "$filename" \ -e "s|ports: # plex|#ports: # plex|g" $filename \ -e "s|- 8096:8096 # plex|#- 8096:8096 # plex|g" $filename fi if [ "$VPN_ENABLED" == "y" ]; then sed -i -e "s;#network_mode: \"service:gluetun\";network_mode: \"service:gluetun\";g" "$filename" \ -e "s;ports: # qbittorrent;#port: # qbittorrent;g" "$filename" \ -e "s;- 8080:8080 # qbittorrent;#- 8080:8080 # qbittorrent;g" "$filename" \ -e "s;#- 8080:8080/tcp # gluetun;- 8080:8080/tcp # gluetun;g" "$filename" fi $dc up -d echo "YAMS was updated and it is starting. Wait 1 min until all the services are up and running..." fi ``` </details>
cooooosh commented 2024-03-01 12:51:20 -03:00 (Migrated from gitlab.com)
Copy Link

changed the description

changed the description
cooooosh commented 2024-03-01 12:57:14 -03:00 (Migrated from gitlab.com)
Copy Link

changed the description

changed the description
cooooosh commented 2024-03-01 12:58:27 -03:00 (Migrated from gitlab.com)
Copy Link

changed the description

changed the description
cooooosh commented 2024-03-01 13:00:20 -03:00 (Migrated from gitlab.com)
Copy Link

changed the description

changed the description
cooooosh commented 2024-03-01 13:00:59 -03:00 (Migrated from gitlab.com)
Copy Link

changed the description

changed the description
cooooosh commented 2024-03-01 13:15:30 -03:00 (Migrated from gitlab.com)
Copy Link

changed the description

changed the description
cooooosh commented 2024-03-01 13:17:31 -03:00 (Migrated from gitlab.com)
Copy Link

changed the description

changed the description
cooooosh commented 2024-03-01 14:27:18 -03:00 (Migrated from gitlab.com)
Copy Link

changed title from Start YAMS in stages for Low Power servers to Start YAMS in stages for {+Testing & +}Low Power servers

changed title from **Start YAMS in stages for Low Power servers** to **Start YAMS in stages for {+Testing & +}Low Power servers**
cooooosh commented 2024-03-01 14:27:18 -03:00 (Migrated from gitlab.com)
Copy Link

changed the description

changed the description
rogs commented 2024-03-01 14:42:55 -03:00 (Migrated from gitlab.com)
Copy Link

Hmmm... I've tested YAMS and the YAMS cli in several Raspberry Pi with multiple containers running and didn't notice failures. There are also a couple of folks over our Discord that run +20 containers with the YAMS configuration and haven't had any problems. Are you sure your containers are configured correctly? We've seen some DNS issues with Docker and some ISPs. It's not too common, but it does happen.

Either way, this looks overcomplicated. I do understand the idea, and your changes look good, but I still don't see how this can be useful for everyone. The purpose of YAMS is to be as easy to use as possible. In my eyes, adding multiple commands to achieve almost the same thing defeats that "simple" purpose.

I'll keep the issue open in case I change my mind. Also, you can always fork YAMS and do the changes on your fork! I would love to see that!

Thank you so much for the suggestion and the feedback. It really helps development!

Hmmm... I've tested YAMS and the YAMS cli in several Raspberry Pi with multiple containers running and didn't notice failures. There are also a couple of folks over our Discord that run +20 containers with the YAMS configuration and haven't had any problems. Are you sure your containers are configured correctly? We've seen some DNS issues with Docker and some ISPs. It's not too common, but it does happen. Either way, this looks overcomplicated. I do understand the idea, and your changes look good, but I still don't see how this can be useful for everyone. The purpose of YAMS is to be as easy to use as possible. In my eyes, adding multiple commands to achieve almost the same thing defeats that "simple" purpose. I'll keep the issue open in case I change my mind. Also, you can always fork YAMS and do the changes on your fork! I would love to see that! Thank you so much for the suggestion and the feedback. It really helps development!
cooooosh commented 2024-03-01 15:13:11 -03:00 (Migrated from gitlab.com)
Copy Link

Thanks for the response! I typo'd this in the initial comment but I've since moved to a more powerful machine and everything spins up and down in ~10-15 seconds, so (un)fortunately I'm unable to further troubleshoot those errors.

I think the other use case (not bringing down core services while trying to implement new ones) is the more compelling one. It's easy (enough) to use docker CLI to tinker with new containers, but I've added these commands on my and instance and it's been nice so far. I do feel like there's a middle area between "easiest possible setup" and doing it all by hand, where a lot of people who are building home servers sit. Doubt I'll make my own fork, but now you've got me thinking...

Anyway, thanks for the discussion! I probably would never have gotten my server up and running without YAMS so I'm forever grateful 😄

Thanks for the response! I typo'd this in the initial comment but I've since moved to a more powerful machine and everything spins up and down in ~10-15 seconds, so (un)fortunately I'm unable to further troubleshoot those errors. I think the other use case (not bringing down core services while trying to implement new ones) is the more compelling one. It's easy (enough) to use docker CLI to tinker with new containers, but I've added these commands on my and instance and it's been nice so far. I do feel like there's a middle area between "easiest possible setup" and doing it all by hand, where a lot of people who are building home servers sit. Doubt I'll make my own fork, but now you've got me thinking... Anyway, thanks for the discussion! I probably would never have gotten my server up and running without YAMS so I'm forever grateful :smile:
MasonStooksbury commented 2024-09-23 00:30:48 -03:00 (Migrated from gitlab.com)
Copy Link

@rogs Can this be closed?

Seems that it's otherwise been resolved

@rogs Can this be closed? Seems that it's otherwise been resolved
rogs commented 2024-09-23 08:50:41 -03:00 (Migrated from gitlab.com)
Copy Link

That's correct, thanks!

That's correct, thanks!
rogs (Migrated from gitlab.com) closed this issue 2024-09-23 08:50:47 -03:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
add-to-docs
bug
done
feature-request
in progress
troubleshooting
v2
yams extra
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
rogs
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: rogs/yams#31
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?