Short answer: it depends...
Long answer: for long prompts it usually doesn't make much difference, but for short prompts the difference can be huge.
For example:
The two images below use the prompt with US spelling: "organized colors."
The next two images are for the prompt with AU spelling: "organised colours."
Key takeaway: Always be experimenting with your prompts, including the language used.