General thought:
- Interesting idea; for most part, I thought it reads quite well. The methods are interesting.
- I will be speaking from a beginner’s perspective — so I will focusing more on what I don’t understand; and therefore needs more explanation / clarity.
- Some assumptions that I don’t understand
- What does LLM values mean? Maybe some examples?
- What are the issues if LLM fail to be “multi-cultural”? Any real-world implications?
- In modern LLMs, I don’t think the system prompt is static? While for LLMs tested and other older LLMs, there is no memory — so the system prompt also does not contain the user’s country. Why do you use system vs. user prompt and not just conflicting countries in the system prompt?
- I don’t get what does Composite Identity, User and System baselines from Section 5. What country-pair are we talking about? Is it different from the methodology?
- Idk what’s going on with Figure 3 and 4.
- How does the Shifted to Sys and Shifted to User points calculated?
- Where are the global south (Myamar and friends)? Why are they not in the Figure?
- I can’t see the “clear upward left movement”, and the “shift in clusters to the lower-right part of the plot”
- I can’t clearly see the Western countries in Figure 4.
- Why are they multiple data points for each country?