I’ve got this command in a bash script:

TEST=$(curl -o /dev/null -s -k -w "%{http_code}" -u "${USERNAME}:${PASSWORD}" "${URL}/dashboard/")
echo "${TEST}" #debug

When the script runs, the output is “000”.

When I run the same curl command from the shell, the output is “200” (which is correct, since the URL is valid).

I verified that the USERNAME, PASSWORD, and URL vars are being passed to the subshell.

I’d appreciate it if you could point out what I’m doing wrong here. :)

UPDATE: This has been solved.

  • Aa!@lemmy.worldEnglish
    5·
    6 days ago

    I love bash’s -x option for exactly this type of debugging. It prints out every command before running it, so you can see what is different about the command as the script is running it

    bash -x my-script.sh

    Off the top of my head, I’m guessing the inline shell command $( … ) is eating your double quotes, and I bet you should escape them, particularly around username and password

  • JoeyHarrington@lemmy.caEnglish
    2·
    6 days ago

    Add echo before your curl command to see what the expanded command line actually is. Perhaps the arguments aren’t as you expect them

    • Mike Wooskey@lemmy.thewooskeys.comOPEnglish
      1·
      6 days ago

      Thanks for the suggestion. I already tried this: echo-ed the curl command to stdout and then in the shell copied it and pasted it to run it. From the script I get “000”, from the shell I get “200”.

  • ElsieEnglish
    2·
    6 days ago

    what happens when you wrap $() in quotes?

  • GenderNeutralBro@lemmy.sdf.orgEnglish
    1·
    6 days ago

    I verified that the USERNAME, PASSWORD, and URL vars are being passed to the subshell.

    But are they being passed into the script? Normal behavior would be for variables to propagate to subshells but not to scripts/commands, unless you explicitly export them (e.g. export USERNAME='my_username' instead of just USERNAME='my_username'). Is that what’s happening here?

    Another possibility is that when the script runs, it is run with a different shell, like sh or zsh. Does your script have a #! line at the top, and is it pointing to the same version of bash you are running in Terminal?

    If you run echo $PATH; which curl in the script vs directly in your Terminal, do they output the same results? This is a bit of an edge case, but I’ve been stymied before by having multiple versions of curl installed (e.g. via homebrew and via my distro) and different PATHs in different contexts causing unexpected behavior.

    • Mike Wooskey@lemmy.thewooskeys.comOPEnglish
      1·
      6 days ago

      Thanks for the suggestions.

      The script is pulling the values of USERNAME and PASSWORD from a .env. I added debug echo $USERNAME $PASSWORD in the script and it shows the correct values, so the script is pulling the values correctly and storing them in the vars correctly. I also added that echo to the subshell command, like:

      TEST=$(echo $USERNAME)
echo $TEST

      …and the result was the correct USERNAME.

      The script does begin with #!/bin/bash.

      echo $PATH; which curl produces identical results when run from the shell and the script.

    • Mike Wooskey@lemmy.thewooskeys.comOPEnglish
      1·
      6 days ago

      Ah!

      TEST=$(curl -o /dev/null -s -k -w "%{http_code}" -u "${USERNAME}:${PASSWORD}" "${URL}/dashboard/#/http/routers")
echo "previous exit code: $?" #debug
echo "${TEST}" #debug

      This outputs:

      previous exit code: 7
000

      Is $? referring to the exit command of the curl in the subshell? Or is it referring to the assignment of the subshell’s output to the TEST var?

      • rwdf@lemmy.worldEnglish
        1·
        6 days ago

        Exit code 7 means curl couldn’t connect to the host, so I would try just curling a URL you know is valid directly, not setting it as an env var, to see what happens then.

          • Mike Wooskey@lemmy.thewooskeys.comOPEnglish
            2·
            6 days ago

            I stumbled on a possible cause, but more background is necessary to explain.

            The script actually creates an ssh tunnel (to the Traefik host) and then does the curl. So the code is like:

            SSH_CMD="ssh -N -L ${LOCAL_PORT}:127.0.0.1:${REMOTE_PORT} ${REMOTE_USER}@${REMOTE_HOST}"
$SSH_CMD &
SSH_PID=$!
SSH_RESULT=$?

TEST=$(curl -o /dev/null -s -k -w "%{http_code}" -u "${USERNAME}:${PASSWORD}" "${URL}/dashboard/")
echo "${TEST}" #debug

            What I learned is that when i run the script, the tunnel is successfully created but the curl fails; but then if I run the script again a second tunnel is created and the curl works fine.

              • Mike Wooskey@lemmy.thewooskeys.comOPEnglish
                2·
                6 days ago

                That seems to have done it!

                Running the ssh -f... instead of ssh.... & seems to work first time and every time.

                It makes it so SSH_PID=$! doesn’t work, but I used pgrep -f <ssh command> instead.

                Thanks!

                • ignoble_stigmas@sh.itjust.worksEnglish
                  1·
                  6 days ago

                  You are very welcome! That was the hypothesis, that ssh doesn’t go into background as you want it to, since it works for the second run, but tunnel is there after the first.

            • rwdf@lemmy.worldEnglish
              1·
              6 days ago

              Are you trying to reach a URL on the same host you’re ssh-ing to? That would create some interesting effects.

              Especially since it works the second time it could mean that the second time you’re actually on the host and ssh-ing to the host itself and then curling localhost.

            • rwdf@lemmy.worldEnglish
              1·
              6 days ago

              Ah, I see. I guess they get different contexts or something? (Edit: I re-read your post and this does not make any sense :)) What if you chain the ssh command and the curl using &&?

              • Mike Wooskey@lemmy.thewooskeys.comOPEnglish
                1·
                6 days ago

                I’m not sure how to chain these 2 commands with &&, because the SSH command is being put in the background with &.

                This doesn’t work:

                SSH_CMD="ssh -N -L ${LOCAL_PORT}:127.0.0.1:${REMOTE_PORT} ${REMOTE_USER}@${REMOTE_HOST}"
$SSH_CMD & && TEST=$(curl -o /dev/null -s -k -w "%{http_code}" -u "${USERNAME}:${PASSWORD}" "${URL}/dashboard/")
SSH_PID=$!
SSH_RESULT=$?
echo $TEST

                Perhaps I don’t need it in the background - the goal was to establish the tunnel and then continue with the script without it hanging until the ssh command is canceled.

          • rwdf@lemmy.worldEnglish
            1·
            6 days ago

            Someone else suggested the env vars arent being expanded correctly inside the $(curl …), which could be the culprit … If a straight up URL works that would indicate that something like that is happening.

            That said, I just tried setting an env var called URL=“” and curling it, and curl said exit code 2, no URL specified, so something else is going on here.

        • Mike Wooskey@lemmy.thewooskeys.comOPEnglish
          1·
          6 days ago

          Here is the result of the script curling a known good URL (it still results in exit code 7 and thus a result of “000”), followed by a copy-paste of the curl command run in the shell (exited with “200”):

          $ ./test.sh
curl -o /dev/null -s -k -w "%{http_code}" "https://i0.wp.com/www.notquitezen.co/wp-content/uploads/2022/07/Is-Happiness-an-Emotion.png"
previous exit code: 7
000
$
$ curl -o /dev/null -s -k -w "%{http_code}" "https://i0.wp.com/www.notquitezen.co/wp-content/uploads/2022/07/Is-Happiness-an-Emotion.png"
200