Writing my very first assembly program
It's not made clear in the tutorial, but the word "register" can mean a couple of different things.
The universe that the 6502 can see and manipulate are 65536 byte-sized memory locations, numbered $0000 through $FFFF, plus the CPU's own internal registers, like the accumulator ("the A register", and yes, A does stand for Accumulator). The accumulator and other CPU registers don't have memory addresses, they're in a special extra place off to the side (they're in... the CPU rather than in the RAM chips).
However, a CPU alone in a memory space isn't very useful, the computer needs input and output to be useful. Often, these extra devices are wired up to the CPU's memory interface, so that the CPU can access them as though they were memory, but they're not just ordinary RAM chips, they *do* something when you read from or write to them. From the point of view of the connected hardware, these things aren't ordinary memory, they're registers (just like the CPU's accumulator isn't ordinary memory) but from the CPU's point of view they *are* memory, because that's how the CPU accesses them.
In the tutorial's emulated computer, $00FE and $00FF are registers you can read from, and if you squint you might say that $0200..$05FF are registers you can write to, but from the CPU's point of view they're all just memory and very different from the accumulator.
All that to say: it's not quite true to say "values are written as `#$xx` while `$xxxx` portrays a register value". Rather, `$xxxx` means "read from (or write to) memory address `$xxxx`" while `#$xxxx` means "the actual literal value `$xxxx`". It's just that in this specific instance, the accumulator is only 8 bits (so there's no point adding leading zeroes) while the destination location needs more than 8 bits to express (so there's not enough leading zeroes to remove).