In this episode of the International Web Effectiveness Podcast, Bill Hunt and Bill Scully take a deep dive into the challenges of IP detection, country and language selectors, and their impact on global website strategy. They break down the role of redirect logic, cookies, and VPNs in shaping the user experience across different markets. They also discuss how businesses can handle IP detection effectively, verify IP databases, and ensure Hreflang tags are properly set up to enhance site performance and SEO.
Key Takeaways
- IP Detection and Redirects: Ensure users can easily switch markets, as IP redirection can confuse those wanting to choose another region.
- Cookies and User Preferences: Use cookies to store user region preferences and allow easy opt-out or changes.
- VPN and Cross-market Purchasing: VPNs may bypass market restrictions, so implement measures to prevent cross-market purchasing.
- Country/Language Selector UX: Simplify market selection with a globe icon or dropdown without flags for better UX.
- Testing International Site Performance: Use tools like Screaming Frog and VPNs to test redirects, verify Hreflang tags, and ensure compliance.
- Analytics and User Behavior Insights: Track how users interact post-redirect to improve UX, such as clicks on selectors or search boxes.
- Avoid Redirecting Bots: Do not redirect search bots to avoid damaging site performance and SEO.
- Practical Tips for Global Site Optimization: Test your site’s international functionality with tools like TunnelBear for region-specific insights.
Catch the Full Episode
YouTube: https://youtu.be/uYDxZKFeZn8
Spotify: https://open.spotify.com/episode/3PEOJYM9gryYvJLUihdT3y?si=ZTpl01u0Txuad2tBYTK-2g
Transcript
Bill Scully: Welcome to episode sixteen of the International Web Effectiveness Podcast, where Bill Hunt and I, Bill Scully, focus on strategies and techniques to improve web presence and manage global websites. In our last episode, titled What is International SEO?, we tackled real challenges like adapting to local search behaviors and avoiding duplicate content penalties across regions. There were a lot of valuable insights in that episode.
In this episode, we’re going to dig into IP location and language detection. When I think of IP detection and language, I’ve seen a lot of sites do it wrong, and some sites do it really well. You have some great examples there. Basically, what we’re talking about is when someone visits your site, their IP is detected, and then they are redirected to the appropriate page mapped to that IP address.
So, Bill, how are you doing?
Bill Hunt: Good. Everything’s good. So, is this something you want to tackle? Want to get started?
Bill Scully: Sure. Yeah. We covered this a little bit in the International SEO session at Pubcon, where people were asking about it. I also encountered it recently when someone I was talking to couldn’t get their hreflang right, so they decided, “Fine, we’ll just use IP detection to route users to the correct place.” I thought now would be a good time to discuss this because, like you, I’ve seen a lot of people get it wrong. There are still many sites that don’t implement it effectively.
It was interesting; as I was writing an article on this topic, I found a few articles that broadly explained how to set it up. Some provided SEO perspectives, while more than a dozen cautioned against using it, claiming it’s the “spawn of Satan” and provides a bad user experience. This goes back to your point about how many people do this wrong.
Bill Hunt: Let’s circle back. As you said, when a user arrives, the detection can happen at the browser or server level. Some even perform it at the CDN level, like Akamai or Cloudflare. This is where issues often arise because it’s treated as a “set it and forget it” feature. Similar to hreflang, people enable it, check the box, and move on without thinking about it again.
Bill Scully: So, here’s what happens: a user from the United States might try to visit a UK website, and the system detects that they’re from the US. There are two ways this is commonly handled. One approach immediately redirects them to the US site, ideally landing them on the same page. However, nine times out of ten, they end up on the homepage of the UK site. This can be frustrating, as they may have come from search, only to land on the wrong page, or they get directed to the homepage and have to restart their activity.
Another common feature is language detection, which is based on browser settings. In my experience, most people don’t realize that downloading Chrome, Firefox, or Microsoft Edge in the US sets the default language to American English. If downloaded in another country, it sets the language to the local language. So, if you download it in Germany, for example, the language will be set to German.
This detection method can look at the languages configured in the browser, which can be prioritized hierarchically. So, if a user’s primary language is Japanese, but they also speak English, they might have Japanese and English set as language preferences. The system then uses these details, combined with IP data, to determine the page the user should be directed to.
One of the primary mistakes people make is relying solely on their CDN or a licensed plugin to handle this. Similar to hreflang, we advise building a country-language matrix. Identify all the sites you have, who you’re targeting, and what language each version should support. Then cross-check these configurations to ensure they align with the actual settings.
Bill Hunt: Another common issue is adjacent markets. For example, if you have a UK site but no Ireland site, where do users from Ireland go? Often, they’ll be directed to the UK site, though some might send them to a neutral /en site. If a company is US-centric, it might route them to the US site. Typically, they’ll end up on the UK site.
But what about countries like Lithuania? Here’s an example we use in one of our training modules. My developer, Dmitri, found what he needed on the UK site through Google, clicked on it, and was redirected to the Italian-language version of the Swiss website. After some investigation, we realized that lowercase “lt” (for Lithuania) looked very similar to “it” (for Italy), and someone mistyped it, resulting in the incorrect redirection. This is why logic needs to be thought through carefully.
At a minimum, you want to redirect users to a site with an equivalent currency. For example, Lithuania uses the euro, so do you send them to Switzerland, which uses the Swiss Franc, or to another market that doesn’t use the euro?
Bill Scully: Or where we need to have a thought process. Now, all that being said, the one thing I want to talk about is some of the reasons people need to do this. A lot of SEOs have a couple of reasons they’ll give without any care or concern about why you should or shouldn’t use IP or language detection. Some will overly interpret Google’s cloaking definition, which says that when you give Google one page and a user another, it defines that as cloaking. They hide behind that to say you shouldn’t use IP detection for this reason.
But if you look at what Google is actually saying, they mean you shouldn’t give a different page. The reality is, you’re not giving Google a different page. If Google tries to fetch your German page, and it doesn’t present a language preference, it won’t take cookies, but it will present an IP, often from Mountain View in California. We see this happen a lot, where Google gets bounced back to the United States. You end up doing the very “bait and switch” they advise against because Google requested the German page, intending to index it. But when you bounce them to the U.S. version, you’re essentially blocking their attempt to do what you wanted.
Bill Hunt: What’s even crazier is when people don’t consider this for their XML sitemaps. I have more than a dozen examples where Google tried to fetch a local market sitemap but got bounced back to the U.S., so they could only ingest the U.S. sitemap. They didn’t get source URLs, couldn’t crawl or index, and then people wonder why their markets aren’t in Google—why they’re not getting traffic or ranking. It’s because nobody thought to put an exemption in for search engines with user agents.
The good news is, if you’re using a Cloudflare or Akamai solution, they’re smart enough to handle this. But if you’re using something from, say, “Johnny’s Plugins R Us” on a WordPress build, and downloaded a freeware version, they’re definitely not going to handle it correctly. That’s one thing.
Bill Scully: Now, let’s go back to other reasons. If you’ve worked with regulated industries—spirits, tobacco, pharma, or defense products—they have to gate their product. You can’t sell or even advertise alcohol in most of the Middle East, so you have to block access from those markets. There’s no negative impact on SEO because those users aren’t supposed to see it anyway. And Google can handle it, too—why would they need a Middle Eastern or Arabic site if it’s illegal? We’ll talk more about gaming the dark market in another episode, but that’s one reason people need this.
Bill Hunt: Another reason is regulatory compliance, like GDPR. If someone comes from Europe, you need to display a privacy statement and allow them to choose which cookies they’re willing to accept. You need to accommodate EU or California regulations. Interestingly, it’s actually against the law to block these users. If you don’t operate in that market, it’s no harm, no foul (and this isn’t legal advice), but if you’re targeting those markets and want to sell there, you need that popup for compliance.
Bill Scully: Another less-discussed reason is contractual obligations. For example, game companies often have market contracts that gate territories. We’ve seen this a lot with markets like Russia and China. Increasingly, brands in private equity deals get exclusive rights in certain regions. If a brand has exclusive rights in France, for instance, anyone trying to access your U.S. site from France may need to be redirected to the French site due to contractual obligations.
Bill Hunt: Then there’s web effectiveness, which we’re passionate about. Load balancing is another consideration, especially with global sites. For instance, a user from France or Italy might get routed to a nearby data center, which is where CDNs come in.
Another key reason for using IP or language detection is product availability and pricing. You may want users to see only the products available in their market.
One final point, which is also regulatory, concerns people on platforms like Shopify. They often build multiple Shopify “pods” for different regions. For instance, all EU sites might be in one pod, while non-EU sites, like those in Latin America, are in another. Each region has its own legal and tax compliance requirements, making it necessary to separate them. These are all reasons why people need to use geolocation to move users around.
Bill Scully: I think to throw in another one, when you’re marketing the domain, if you’re just putting the domain up, there’s no indication as to what page you should be looking at. So, there’s a lot of valid reasons to do this. I think that’s a brilliant one—if you’ve got a dot-com, and there’s nothing to designate the region or language.
Bill Hunt:
If you listen to Fabrice from Bing (and I think I mentioned this in a previous podcast on international SEO), I’m starting to lean toward his idea of language-specific sites. We can now use modern technology to manage these. For instance, if you have a French language site, there’s no need to write in and tell us there’s a difference between French for Canada, French from Morocco, and French from France. Yes, we know that. If you can and are able to create dedicated sites, that’s the right thing to do. But if you’re just starting out, testing the market, and funds are limited, you’re probably not going to do that.
What you can do, though, is adjust where leads go based on the user’s location. For example, if someone visits your French site from Morocco, you could adjust your lead generation module to send those leads to an EMEA contact who handles that region. Similarly, a visitor from Canada could be directed accordingly. Another important reason to consider this setup is security and fraud; some markets are notorious for fraud and other suspicious activities. Using IP-based geolocation, you can at least direct such users to a different channel.
Bill Scully:
If you’re running a campaign and want to use a campaign slug, you don’t need one for every language and country. A URL like dot-com/campaign-name can work as the main link for print promotions. The same principle applies when setting up QR codes.
In the article we’re going to post, there’s an example of localized offers. For instance, if you go to Orlando, Florida, and search for anything related to entertainment or tourism, you’ll notice local-only discounts. If you’re an Orlando resident, you might get a great deal on a Disney season pass. People within the Orlando IP range receive those offers, while others outside the region don’t see them.
Bill Hunt:
What are some things people get wrong? We’ve covered a lot of benefits, but what about the risks of not doing it right? For one, people travel, and they may not know the local language. Have you ever worked on a site where, every time you tried to go to a page, you were redirected to another? It’s frustrating for users, and unless something like Hreflang is properly implemented, it can make the experience difficult.
If you’re going to implement geolocation, the biggest risk is frustrating the user. For example, if you’re in Paris for the Olympics and you try to book something for your home location in New England—say, snowshoes—but are redirected to the French site, you might not speak French. You’re likely not going to buy something from the French site and have it shipped to the U.S. A common example is tourists in Japan, who often struggle to access non-Japanese sites because of automatic redirection.
Bill Scully:
Smart companies offer a choice: they might display a pop-up saying, “We detect you’re in Japan, but would you like to go to Lands’ End in the United States to shop?” This approach gives people the option to go where they want.
Bill Hunt:
This ties into something we’re going to wrap up with—measuring the experience. I’ve asked numerous companies if they track the results of offering that choice, like “Do you want to continue to the French site or go to the U.S. site?” How many people say, “No, I’ll stay on the French site” versus “Yes, I want to go to the U.S. site”? I haven’t found many companies tracking this.
Additionally, if you implement IP detection, it’s essential to measure the journey. Did the user who chose to stay on the French site become a customer, or did they abandon the process? Tracking whether users are routed to the correct place and how they respond to that experience is crucial.
Bill Scully:
Right. So one is giving users a way to get out and go to where they actually want to go. The option, you know, between the U.S. and France, for instance, is good. Then, there’s the other aspect we talked about earlier—market adjacency or relevance.
Bill Hunt:
Another big consideration is language preference. For example, if my preferred language is English, but I’m in Germany, and there’s no English-language site available, I’m directed to a German-language site. How can I interact with that site? If you’re using Chrome, a translation service might pop up, but there are still issues.
Bill Scully:
Have you seen this happen on a mobile device? I haven’t noticed it on mobile; on desktop, it obviously pops up with the option. I’ve seen the localization prompt there. Now, I think in Chrome on my Pixel phone, I can select an option to translate the page. It just requires me to go in and manually choose that, but it works as a backup option.
Bill Hunt:
Another risk related to our adjacent market scenario is what to do with people from countries outside of the main regions. Where do we send them? This question aligns with the x-default issue. Where do people in Fiji, Tonga, or Sierra Leone go? The internet is an almost country-agnostic space, where people can find you from anywhere in the world. The challenge is ensuring they can access what they need when they find you.
Bill Scully:
Our closing recommendation would be that, if you haven’t already, you might consider conducting an IP or language detection audit. If this is a long-term strategy, two things are essential. First, measure it. You’ve always advocated for measuring everything you can as long as it leads to actionable insights.
Bill Hunt:
We need to examine whether this provides a good user experience. For example, when users come through a gateway or there’s no gateway at all, what is the bounce rate? We analyze this with our paid and organic ads, and we mentioned in a previous episode that there’s an 87% bounce rate in Latin America and Asia. Much of this is due to users landing on the wrong market site.
Are things improving? Are you getting negative feedback from users being routed to the wrong places? What’s the user behavior once they’ve chosen their path? Are they progressing at an acceptable rate through the site? These are important factors to monitor.
I recently spoke with our good friend Partha, an analytics expert. I asked him how often he’s seen clients monitor this kind of user behavior. Surprisingly, he said that in his entire career, only about ten clients have conducted a detailed analysis on user behavior and feedback, or even a simple A/B test on gateways versus direct routing. That’s quite low for something this critical. It’s similar to a store manager forgetting to flip the “Open” sign on the door or forgetting to update business hours in Google My Business, which says you’re closed on Saturday when you’re actually open.
Bill Scully:
So yes, I think people need to consider this holistically.
Bill Hunt: Now, moving on to internationalization and getting users to the correct location, I’m a big fan of allowing users to know where they are and giving them options to navigate elsewhere. Many sites, for example, place a dropdown in the footer for this purpose.
I’ve got to say, I don’t know that I’ve ever realized I was on the wrong site without checking the URL. Just the other day, I saw a site that actually had, in the hamburger dropdown at the very bottom of the main menu, a flag and selector. I thought that was slick, especially for people who might not realize they’re on the wrong page.
What else was I thinking? Oh, GDPR. If you’re collecting IP information now, you need to include that in your privacy policy.
Let me ask a question: if you were starting from scratch with a site and could get Hreflang set up correctly, how strict would your IP redirects be? I think I would be moderate on this. I’d definitely have a left or right choice pop-up for users because I travel a lot and encounter this frequently. Being moderate seems ideal.
We use it because, if we’re in any market with a legal requirement, we want to comply. So why not do it right and maximize the benefit? Anyone with even a minimal presence in the EU is essentially required to do this, so it’s best to set it up correctly.
As I recommended earlier, I’d start with a country-language matrix: list where you’re operating now, then expand to the G20, G50, or top 50 countries. Check your analytics—there’s a great Looker Studio plugin that shows where visitors are coming from by market. Even if you don’t have a market-specific site, it’s good to track visitor behavior and see where users are directed.
If you’re not an international traveler, there’s actually a function in Chrome Developer Tools that lets you set locations to test redirects. You don’t necessarily need a VPN, but if you travel frequently and want to watch Netflix or other location-based content, you might already have one. Tune your VPN to different markets, ping it, and see what happens. Use incognito mode, too, because…
Back to your point about cookies and IP collection—when you cookie someone, make sure they can override it. I shared an example at Pubcon about a big company. When you accepted their cookies, they locked in your country and language preference. When I tried to access their German page, not only was I blocked, but Cloudflare showed an error. First, it gave a 403 error, then a 503, signaling a server issue.
That was misleading because the system blocked access based on my IP and cookie settings, without any option to override. Even with my VPN set to Germany, I couldn’t get in until I used incognito mode, which removed my cookies. Once I did, I could access it. This company hadn’t considered this issue, and it’s why audits are crucial.
So, let me throw out a scenario: there could be an automatic redirect for a .com or an unknown URL, so users end up on the correct page. Once there, if their IP doesn’t match the region, a prompt could ask, “Is this where you want to be?” That way, users aren’t auto-redirected every time they land on a page; they have the option to navigate to other regions with cookies.
What would you do with the cookie after they select? If I’m trying to access a different market, I’d go back to a default rule with a pop-up. If someone really wants to go to Germany, the prompt can confirm it and offer to set it as their preferred location. We need to think through scenarios like these carefully.
Now, if you cannot sell, cross-market again because, right, people do this. They’ll turn on their VPN, and then, like, I know EA Sports and some of the other game creators and resellers have this problem where a game might be, I don’t know, I’m not a gamer, but the prices may not be right. Let’s say fifty bucks in Market A, but it’s seventy-five dollars or seventy-five euros. So people, you know, like, “Oh, crap, I’m gonna change my VPN, go to that market, and buy it.” I might have to pay a little extra in shipping, or if it’s a digital product, then I can get it. So, a lot of times, that’s why the cookie, and this extra level of prevention, is there to stop those people from buying outside of their market.
And so, that makes sense, right? And again, the country-language matrix, and let me add another name to it: the rules matrix, where in-market does this, in-region does this, outside of region does this, and you build those in. So, like that scenario, if I came in and tried to go to your site and I’m cookied for another market, it at least asks me, “Do you really want to go into Germany?” And you can be as specific with that as you want, then say, “Do you want to update your preference? Yes or no?” This is like IBM and HP, I think Lenovo will do this. If you try to go cross-market with a cookie, it says, “Do you want to update your preference?” But this company is like nothing. And then, it was just brute force, you know, error message that you’ve broken something. And so, that tells me, and in their case, they didn’t have any logic other than, “He’s cookied for here, that’s where he needs to go.” And we don’t run and write a bunch of custom messages, so we’re gonna pop up the “you denied” error: a soft 403, instead of saying “Denied,” and then that cryptic twenty-seven-digit code from Cloudflare.
And there was no good reason to deny you, right? There was no good reason. Other than, when I asked them why I was denied, they said, “Because your cookie preference was for the United States.” Because that went away. Now, it didn’t cookie me when I was in incognito. So, there was no cookie, and it didn’t cookie me. It popped up, saying, “Do you accept it?” I just said no and was able to go into Germany. Again, I had to be on the VPN, so I bypassed all their infrastructure. Then, they were like, “We had never considered this being a problem.”
Here’s another thing that a lot of big companies don’t realize: A lot of times, they’re on a network outside of their market. Have you ever encountered this? I bet, like Siemens, for instance. I had somebody three or four months ago, they’re like, “This is what I’m seeing.” I’m looking at their screen, and I’m like, “You’re on a US IP.” And because everything’s in the United States, they’re like, “You know, I’m sitting in Switzerland.” I’m like, “You’re on a US IP.” And they’re like, “I’m not using a VPN.” So we pinged their IP, and sure enough, it was the US and it was listed as their HQ. So, they were tunneled into their headquarters and then, you know, out into the US internet. Wow. And they’re like, “I had no idea that I was being tunneled to another market.” And so, sometimes, you know, you’re not seeing things like that.
Another thing, talking about CDNs, I remember doing a couple of projects in Italy where the company that set up the websites and was putting this in place didn’t realize that because of the telecommunications laws in Italy, nobody had CDN representation in Italy. It was in all adjacent countries. So, the nearest sort of, what is it, host, if you will, the optimal host, was like in Spain, or it might have been here or there. It could have been in Switzerland, but it wasn’t in Italy. So, they were getting a lot of issues with their IP detection for people in Italy. So, you know, what we had to do was basically say, “Okay, we’re gonna have to rethink this.” And when we know somebody’s in Italy, we’re not gonna route them; we will route them to load balance them, but we need exceptions for some of the other things they were doing, because they originated from an Italy IP.
Okay, so again, this is like Hreflang. It’s like, you know, getting consistent URL structures. It’s like centralizing your product information. It’s like, you know, this is equivalent to a problem for people as almost any of the other topics we’ve covered thus far in the podcast. But it is one of these things that falls into that magical set and leaves it. And I’m just curious, of the people listening to this, how many people don’t know, you know, what’s being done.
So, you know, just to wrap up my thought, the three or four things – and we have a post that we’re putting up on, you know, the things you should consider for an audit – is, you know, verify the IP database, you know, do that periodically, and review that with the provider.
I’ve never found a tool—and I think I mentioned last time—that I’m desperately trying not to build, where I can put a bunch of markets in, and it does the IP and pings it. But, you know, you can fake it till you make it with a VPN. You know, an incognito window really helps you look at your redirect logic, see where people are actually being sent, and check what’s being presented to them.
Then the other big one is to make sure you use Screaming Frog or any tool of your choice on a VPN. See if, when I request my XML sitemaps from another market, I can get them. And, you know, again, like you mentioned a couple of times, make sure you’ve got your Hreflang tags correct, so that the right page shows up from the beginning in the search results. Also, look at the SERPs to see what’s being presented.
Bill Scully: What would you do with the cookie after they select? If I’m trying to access a different market, I’d go back to a default rule with a pop-up. If someone really wants to go to Germany, the prompt can confirm it and offer to set it as their preferred location. We need to think through scenarios like these carefully.
Bill Scully:
Another big one, like you said, is making sure you’ve added the appropriate language to your compliance and privacy statements. If you are using this, you have to declare it. And you need to have consent management reviews in place, allowing users to opt out.
The last point you made, when we were looking at Hreflang effectiveness about a year and a half ago, we found that about ninety percent of the thousand most global sites had removed their country selector—like the globe—from the top nav. About half moved it to the bottom navigation (the footer), and the other half removed it completely. That’s your point, right? People had no idea how to get to another market. How would they? You wouldn’t, because you’re being IP detected.
I reached out to a couple of these companies when we found this. We asked, and their answer to me was, “Well, we use IP detection, so we’re confident we’re sending them to where they need to go.” That was the key phrase: “where they need to go,” not “where they want to go.” So, it’s, “We’re confident we know where they are.”
Bill Hunt:
Exactly. And it’s interesting how many creative and UX people despise the country/language selector. I understand—people hate the flag. For example, using an American flag to represent English for another market, or the British flag, or even the hybrid American/UK flag to indicate English. You know, just don’t even put a flag. Simply put the name of the country and the language, or just say “English,” or just put a globe.
Bill Scully:
Exactly. Just put a globe and let people get a country/language selector, then let them navigate from there. If somebody came to the page and clicked that, you’d get better insights. And again, let’s go back to analytics: What percentage of people you’ve forced into a market actually clicked your globe?
Second, how many people that you forced into a market went and clicked on the search box? Because you took them from the exact page they wanted in the search results and dumped them on the homepage in the market you wanted them to go to. What are they going to do? They’re either going to hit the back button and go through the same exercise, or they’re going to go to your search box or pick the globe. So, how many people, when you push them through “door number B,” actually go to one of those two things? That’s, to me, a really, really good indicator that the forced movement you’re doing is not effective. How do you adjust from that? Look at the data and figure out a way to fix it in a way that makes sense for users.
Bill Hunt:
Yep. And I would say that the worst thing you could possibly do is redirect bots. All of those search bots? That would kill your site. So, make sure you know what you’re doing. Bill and I can help with thinking through this whole strategy and testing it, and maybe even implementing it if that’s something you need.
Bill Scully:
One last thing: I know a lot of people I speak with start talking about VPNs. You can get a TunnelBear account for free. It’s really interesting to see the different search results. Take a look at what things look like in France—they look nothing like the US. It’s possible you might have IP redirects in place and not even know it, because you’re not being IP redirected—you’ve been going to the same site the whole time.
Bill Hunt:
Alright, great. Well, we hope that you’re finding value in these conversations and that you’re enjoying the show. We’d really appreciate it if you could take a moment to rate and review the podcast on your favorite platform. We also want to hear from you. Your feedback helps motivate us and keeps making these things better and better. Follow International Web Mastery on LinkedIn, Facebook, and X.
And we want to thank you for tuning in. We hope this episode gave you some actionable insights to improve your website’s efficiency and make your customers more satisfied. Be sure to join us next time as we continue exploring more strategies to increase your web effectiveness.