Can you have a variable in PowerShell here-strings?

Yes.

Review this link before you begin.

You can have a variable in Here-strings but you need to define the variable in the same session. Here is how you do it:

Declare $hello as follows:

$Hello={@"
Hello $x
"@
$x="Jay"}

In order that the variable is expanded run the following:

&$Hello
(This is not case sensitive, you can as well run, &$hello)
and you will get the response:
Hello Jay

(hello or Hello: not case sensitive)
Variable declaration can be in the beginning also as shown:

PS C:\Windows\system32> $Hello={$x="Jay"
@"
Hello $x
"@
}
PS C:\Windows\system32> &$hello
Hello Jay
PS C:\Windows\system32>

Note that defining the variable in single quotes will not alter the response.