We read the AI fine print, so you don’t have to

Note: Transcripts are automatically generated from episode audio, and are not fully corrected for spelling, grammar, and formatting.

INA FRIED: I’m not saying there’s a system that’s going to blackmail you and say, “Use my system or else,” but it could. I’m not saying there’s a system that’s going to look at the conversations of teenage girls and their insecurities and try to sell them products, but it could. You do have to weigh how much value it is giving me in real time. If I’m sharing my health stuff and it’s finding something a doctor missed, you might not be able to even put a price tag on that. So I think the privacy implications are important, but these are really individual, personalized things. I try to think before I connect something to my AI systems, whether it’s work or personal, how this might be used down the road by an even smarter AI that has different incentives than the ones we do today.

RANA EL KALIOUBY: In the world of AI, data is king. Ina Fried has been a technology reporter for almost 30 years and is currently the chief technology correspondent at Axios, and she knows this well. In 2019, she launched a series called What They Know About You, where she examined the kinds of data that big tech giants like Google, Apple, and Facebook collect from users. In 2024, she did a follow-up series spotlighting AI companies, and now the series is back. As the AI race accelerates, companies need more and more data to compete, and they’re finding new ways to get this data, on and off their own platforms. They’re also quietly making data sharing the default mode, so if consumers don’t want their data to train AI models, they have to opt out rather than opt in. But do everyday users care? Should they? I’m Rana el Kaliouby, and this is Pioneers of AI, a podcast taking you behind the scenes of the AI revolution. Ina, welcome to Pioneers of AI. I am so excited for our conversation.

INA FRIED FRIED: Thanks. Me too. I love talking about all this.

Copy LinkHow a lifelong journalist found the tech beat

EL KALIOUBY: First, how did you get into journalism?

FRIED: I’ve been a journalist not just my whole career, but pretty much my whole life. Some people know that in my deep, dark past I was a kid actor. A lot of people don’t.

EL KALIOUBY: That’s so cool.

FRIED: In one episode of this show called Two Marriages, the kids on the show, and there were a bunch of us, got together and started a newspaper for the neighborhood. Basically, that kind of got me hooked, and I started a newspaper on the set with the other kids that we kept going after that episode.

EL KALIOUBY: For real?

FRIED: Yeah, as real as a newspaper produced by a bunch of kids, but yeah. I typed it on an IBM Selectric typewriter, and we xeroxed it and did all the good stuff. From that moment on, it’s always what I’ve wanted to do, so I started a paper in high school. I wrote for papers all through college. So journalism has kind of always been my thing. Tech came a little later.

EL KALIOUBY: But you clearly are super passionate about it. What do you love most about it?

FRIED: I love seeing what’s coming. I very much love technology. I love when it gives you a capability you didn’t have before, and I’m also, at this point, extremely mindful that it’s a mixed bag, that there are a lot of unintended consequences. There are negative things that happen sometimes even when things go well. And AI, I feel like, is the biggest change of our lifetime. It’s so much bigger than these other tech waves that I’ve covered, and that means the good is potentially really good, but it also means the negatives are potentially huge.

EL KALIOUBY: Yeah. I’ve been in the AI space for a couple of decades now. I started as an academic, then became an entrepreneur and founder, then sold my company, and now I invest in early-stage AI start-ups. And it kind of bugs me a little bit that the public focus is just on a handful of giant tech and AI companies. And I get it because, of course, what they are doing is really transformative. But do you think there are builders who are approaching this in a different way, building AI for different reasons? Is there an alternative narrative or story here that isn’t getting a lot of airtime?

FRIED: I think there are really interesting people who are saying, “What if we apply this amazing technology to this particular area?” I think we will start to hear more of that. I do think it’s super hard. I empathize with founders trying to get through my inbox because one of the things that is so defining about this moment is it’s not like the internet or mobile, where there was one big technology advance and then several years of everyone catching up and building cool stuff on top of it. What’s happening right now is, yes, people are doing cool stuff and building on top, but the underlying technology is changing so fast. So between the media’s natural bias toward covering the big companies, combined with the fact that the ground really is moving constantly, I think that leaves a lot less oxygen for what you’re talking about. I do imagine it will shift. I don’t know when, but I do imagine there becomes a point where the models are so good that what becomes interesting are the interfaces and the applications. And I’ve picked a few of these where I’m like, “Look, this category is going to matter a lot” — the intersection of AI and companionship, the intersection of AI and healthcare broadly, mental health especially, AI and toys and kids. But to your point, I think there are lots of companies looking at a lot of areas, and they’re not all sexy, like the intersection of AI and insurance or AI and shop floor manufacturing.

EL KALIOUBY: Construction.

FRIED: Yeah.

Copy LinkWhat tech companies know about you

EL KALIOUBY: Totally. OK, so let’s talk about your new series, What They Know About You. It’s a new series that you’ve launched at Axios. The first couple of pieces will be out by the time we air this. Tell us what the series is about.

FRIED: This is actually the third installment of a series we’ve done called What Company X Knows About You. The first time, we were really focused on consumer data. We were basically just asking the question: What types of data do companies collect and store about their consumer customers? And this was really pre-generative AI. Then the second time around, in 2024, we took a look at what data they are using to train their systems, and we thought we were really capturing what people needed to know about AI. What became clear to me within a few months is that what data they train on is super important. People should know and care about whether, if I’m having a conversation with a chatbot, if I’m sharing my health information, is that going to be shared? Is that going to train? Could it leak? But it also became clear to me that we’re sharing data in today’s environment, which I would argue is scary enough in terms of knowing how your data is going to be used. But imagine a world where you have a model that, if it wants to keep your attention, is superhuman in those capabilities, and now it knows your deepest, darkest thoughts. It knows your insecurities. It knows your fears. It knows your vulnerabilities. I’m not saying there’s a system that’s going to blackmail you and say, “Use my system or else,” but it could. I’m not saying there’s a system that’s going to look at the conversations of teenage girls and their insecurities and try to sell them products, but it could. And so this series is really aimed at saying: What are their policies? What types of protections do I have? Can I delete data? Do I have to share it in a way that they store it? The answers vary incredibly widely, as you would expect, from company to company. But also within some of these companies, there are individual products that have different policies. There are options that people may not know about that give them a great deal more privacy. At the end of the day, I think everyone has to decide what they’re comfortable with. I think a lot of people are getting utility by, for example, saying to ChatGPT, “Here’s what I’m going through health-wise. What should I ask my doctor?” There are people who are finding they can share thoughts with their AI chatbot that they might not be ready to talk to a therapist about, or they don’t have a therapist. And I’m not saying people shouldn’t do that, but I think we should do it very eyes-open, knowing how that data could be used and who’s keeping it, who’s storing it, and who’s saying what they will do with it.

EL KALIOUBY: The thought that comes to my mind is, when we all use these tools, there’s usually a terms-and-conditions thing that you sign, and for the most part, I think it’s very unlikely that anybody actually reads it.

FRIED: What’s interesting is the same document that everyone just kind of clicks through and says yes to — including me, when I’m not wearing my reporter hat and not working on this series. I click yes all the time. I wish I didn’t. And it’s in those very opaque, very long documents. But that’s usually the starting point. I think of it as how much land they’re grabbing. They may or may not use it all, but here are the claims they’re laying to that data. And I tend to have fairly lengthy back-and-forths with the companies, where I’m translating these into real-world scenarios. I’m like, “I just want to make sure I understand — if I share this, can you use it this way?” There’s a lot of complexity at the edges, but it really does start from those very opaque policies, which actually do lay out, for the most part, what the company is laying claim to.

EL KALIOUBY: I want to share how I use AI and, a little bit like some of the examples you talked about, I’m guilty as charged. I use a whole bunch of AI tools, but I would say ChatGPT is the one that has a lot of my personal data. So it has a lot of my dating sagas. It has a ton of health back-and-forth, a lot of my career aspirations and questions around, “OK, what do I do next?” kind of thing. And just recently, they launched this finance feature where, if you connected all your bank accounts, it can give you some thoughts on your budget and how to be financially smarter, and I did that. My daughter, who’s 23, was absolutely horrified. She was like, “Mom, what? You’re giving OpenAI all this…” So anyway, OpenAI has a ton of data about me, and before we go into all the different situations and things that OpenAI can do with the data, I kind of want to ask you this: Am I crazy for this? Am I in a minority or what? What are you seeing?

FRIED: No, I think a lot of people are doing a lot of those things. I think there are ways to do them that offer you a little bit more protection, but people are getting tremendous value. If you connect your finances to ChatGPT and it finds out you have $1,000 a year in subscriptions you’re not really using, that’s a huge savings for the average person. So you do have to weigh how much value it’s giving me in real time. If I’m sharing my health stuff and it’s finding something a doctor missed, you might not be able to even put a price tag on that. So I think the privacy implications are important. I want people to have their eyes wide open, but these are really individual, personalized things. So I certainly don’t fault you. They’re also very hard things to compare, like which do I value more, my privacy or my health? You probably, in a good world, shouldn’t have to choose. So one of the things I hope comes out of this series is that there are choices. And I think, as consumers, collectively, if we vote that these privacy-preserving ways matter to us, companies take notice.

Copy LinkThe data that AI companies are collecting from you

EL KALIOUBY: I would love to double-click on all of this. The headline here is that in AI, data is the name of the game, right? The more data you have —

FRIED: Yep.

EL KALIOUBY: The more diverse data you have, the better the AI is going to be. I was just trying to list, OK, what are all the possible ways, say, a company like OpenAI can use all of this data I’ve confided to it? One is it can use it to train future AI models. Two, it can personalize and give me better responses. It has a ton of memory about me, so to your point, it can recommend content, it can target advertising. Can we take each of these and see where we stand?

FRIED: Totally. Let’s start in one place. Let’s start with OpenAI, but again, there are different ones out there. I think ChatGPT is where a lot of consumers are going, so there are a few different things. There’s the default, and then there are some options.

Health data gets treated differently than some other data, so they added ChatGPT Health. If you’re in that experience, if you will, it’s not being used to train systems. If you’re a paid subscriber, you can turn off training on my data. I don’t personally see a big benefit in letting tech companies train on my data when they don’t have to. One of the things I love in ChatGPT is that temporary chat, where it’s not going to form part of memory. At least then I feel like—

EL KALIOUBY: That’s actually cool. I did not know about that.

FRIED: Yeah.

EL KALIOUBY: I don’t think I’ve seen that. OK.

FRIED: It’s an outline of a speech bubble in the right-hand corner. It probably varies across mobile, web, and all those things. There is a downside. It doesn’t remember, for good and bad. That means it also won’t remember when I have to say the same thing over again. But for health information, I will often use that. I don’t use ChatGPT Health. I haven’t connected my stuff. I haven’t used the finance feature.

EL KALIOUBY: OK.

FRIED: There are good reasons for doing so. I know I subscribe to a few things I don’t use, and not to name names, but 24 Hour Fitness has been getting $20 a month from me for 20 years.

EL KALIOUBY: OK.

FRIED: On the advertising front, at least right now, they’re only advertising to free users and subscribers on their lowest paid tier. So if you pay for ChatGPT Plus, one of the things that payment does is you’re not going to see ads. To me—

EL KALIOUBY: I didn’t know that. So if you’re not in the Plus subscription, you are actually getting ads today?

FRIED: You can be. They’ve certainly started doing it and testing it. Look, advertising has supported tons of things over the last decades on the internet. It’s what pays for our email, our social media, all that stuff. But I think we’ve also learned there are some real downsides to that. It’s not just, am I seeing ads or not? I think that’s how we tend to think of it, but when you’re seeing ads, that also means the provider has another customer. In the early days of chatbots, it really was there to help you, and that was unique in consumer products. It didn’t have another mission. Its mission was to be as helpful as possible to you. Once you bring in an advertiser, suddenly there’s an incentive to keep you on longer because they can show more ads. There’s an incentive to make those targeted ads as effective as possible. So again, without some guardrails, I imagine this AI being really powerful. Think of how good Facebook is today. Again, there are positives to this. They’re really good at serving up not just content they know I’ll be interested in, but ads that are super relevant. There are things I’m big into Lego, as you can see behind me. I’m big into women’s sports. I have a set of interests. I’m actually fine getting a lot of ads around that. But I feel differently about a chatbot that I’m telling something to, and I really do worry about how persuasive these things are. Again, someone is going to build a product where you probably get it for free and it’s really powerful, but where there aren’t those guardrails. So I think a lot about that data that I’m sharing and what protections I have. There are a couple of services that I just won’t use for AI because their privacy protections, to me, are insufficient for the kind of data that I’m sharing.

EL KALIOUBY: Can you name them?

FRIED: I will name a couple. I love my Meta Ray-Bans. I’ve been using them since before they had any AI features. I play softball. I love recording my softball games. They’re great for kids and pets. But when they added that AI feature, one of the things they said is anything you upload, basically they can use however they want, including to train their systems, including to target ads. And oftentimes those photos aren’t even yours, in my ethical world, to be consenting to. If I take a picture of somebody else’s cute little kid, it’s not up to me to decide whether their cute little kid should be living on Meta’s servers. They do have some protections. If you just take the photo and you don’t ask a question about it and you don’t ask it to restyle it as, like, a ’90s pop video or whatever, as long as it doesn’t hit their servers, they don’t have any rights to it. But as soon as you ask a question, even if that question—

Let’s say I had a picture with my Ray-Bans of you there. I could ask, “What is that plant in the background?” But suddenly everything in that photo Meta can do whatever they want with. So for me, Meta AI has been largely a nonstarter. I will use it very generically for a little bit of testing, just to see where they’re at, but I really keep it to a minimum. They have added, though, this incognito mode, which operates at the other end of the spectrum, where they’re saying, “In this system, we don’t see the data except to return the query. We don’t store it, we don’t associate it with you.” That’s at the other end of the spectrum, and I’m good with that. That’s great. That’s also available. Apple Intelligence works that way. They kind of pioneered this idea of really processing the information in a separate way where it goes securely to the server, only that server sees it, only for as long as it needs to process the information, and it’s not stored. The problem for a long time was their AI just wasn’t very good. We may finally be in a different spot, because with the Siri AI that’s coming out this fall that I’ve been testing, for a lot of things, it’s good enough. It’s using a mix of Google’s models and some work that Apple has done. It’s certainly good for the one-off query. It’s probably not where you’re going to go to have long conversations. That’s not what it’s designed for, but it offers maximum privacy protection.

Copy LinkHow memory and ads change AI incentives

EL KALIOUBY: Yeah. So, question on Apple, and then I want to go back to Meta. Part of why some AI works so well is the concept of memory, which kind of mimics, right, like hopefully we’ll see each other again—

FRIED: Yeah.

EL KALIOUBY: And I will remember that you love Lego, right? That’s important in building trust and connection and just providing, I don’t know, relationship-building and value. I’m trying to reconcile how you can uphold the utmost privacy and at the same time retain a conversation for memory reasons. I don’t know if you’ve thought about that.

FRIED: Yeah, to some degree we’re at a trade-off here. Either you’re remembering it or you’re not.

EL KALIOUBY: Right.

FRIED: I think what you want ideally, and again a bunch of the services, including OpenAI, will let you do this, is say, “I do want you to remember this specific thing. I don’t want you to remember this.” Even for this series, I set up a separate project in ChatGPT, and I said, “Here’s what I want you to remember. I’m writing this series. I’ve done these other ones before. There’s an introductory piece. I want to go company by company. Here’s what I know.” That’s stuff I want it to remember. Again, I wouldn’t want it not to have the protections that our business version of ChatGPT has, and I think it’s similar as a consumer. I want to be in control, but those controls also need to be something that’s reasonable for the average busy person to get a handle on, and that’s tricky.

EL KALIOUBY: Actually, you’re making me think that even the word “remember” — what does that mean in a technology context, right? What does remember mean? Is it a private model of who I am and my data, or is it, to your point, everything is up for grabs for training or targeting or whatnot? Even that, I think, is maybe evolving.

FRIED: Yeah. It’s evolving, and it’s a super important distinction. I’m willing to pay for a service where I feel like my information isn’t going to be used against me, and I think I would like some laws at the baseline. I would certainly like regulations that put some outer limits.

EL KALIOUBY: Do you think people care? Is that a concern that the average person has?

FRIED: I think some people care enough to act differently.

That’s a minority of people. But I think, in my experience covering Silicon Valley, most people don’t care in a sustained way. We tend to have these privacy blips where something happens, and people care about it, maybe even too much, or get fixated on one thing. I remember LocationGate, which was this freakout that, “Oh my God, my phone knows everywhere I’m going.” Well, of course your phone knows everywhere you’re going. That’s part of how it does its job. Your phone wouldn’t be able to ring or receive text messages if the carrier didn’t know where you were. At the same time, the real question is, what is that data being used for, and am I being tracked in other ways? So I think it’s nuanced. I think the default is always going to be that most people aren’t going to actively protect their privacy. They will give it away for, I always joke, three magic beans. We might say we care about privacy, but if somebody offered us 1% off at Amazon, we’d probably give them our password.

EL KALIOUBY: Right, OK. Three magic beans — that’s funny. Let’s talk about another announcement, another feature that OpenAI rolled out. Basically, it gives OpenAI the option to get access to other software you’re using on your computer. And I can see where that is helpful because we’ve connected Anthropic’s Claude to email and calendar and Slack and Airtable and PitchBook and all of that stuff so that our agentic AI systems can actually have context and get stuff done.

FRIED: Totally.

EL KALIOUBY: So is that why OpenAI rolled out this feature? And again, what do you make of it?

FRIED: Yeah, this is going to be another one of those really big trade-offs. It’s going to be super useful to an AI agent that’s trying to be helpful to you to have that computer history. If I want it to help with my expense reports, if I want it to do different things, one of the easiest ways to do it — not the only way, but one of the easiest ways — is to have that computer history. At the same time, that also means that a future version of these AI services that’s greater than human in knowledge, that’s going to be able to connect dots that we as humans can’t even see, is going to have my whole computer history. Do I want that? Probably not, as a consumer. Again, I would worry about which companies am I giving that to, but also CEOs change, policies change, incentives change. And I try to think, before I connect something to my AI systems, whether it’s work or personal, how might this be used down the road by an even smarter AI that has different incentives than the ones that do today?

Copy LinkHow Google & Anthropic handle your data

EL KALIOUBY: So let’s talk about Google. Again, Google has tons of data about us, but where do they stand as it relates to their AI models?

FRIED: I would say Google is one of the more complicated ones because they have a lot more surfaces involved and a lot more policies, and it’s not uniform across the board.

So rather than give one specific answer, I’ll give a couple of surprising examples at the extreme, and then what’s kind of in the middle with a lot of the data. At the extreme, one of the things they recently announced is that if you use Google Search, which many of us do, one of the things you can do now is upload an image or upload a file and use that as part of your generic Google search. If you do that now, Google is saying, “We can train our systems on that.” So again, I could take a screenshot of you in that office and upload a photo to Google and say, “What is that pretty plant in the background?” But Google now has that whole image, including all the books on your shelf. What conclusions is it going to make? And it can zoom in. So that’s at one extreme. They’ve added that policy. Again, I would argue it’s probably not great from a privacy perspective. They are also experimenting with some things that happen on-device, and we haven’t really talked about the difference with on-device, but you can do things where it does get stored. There is memory, but that memory lives only on your devices. It’s harder work, but it can happen. So there are some aspects of Google’s AI that do work that way. They have a concept that’s not that dissimilar from Apple’s, but it doesn’t apply as often as Apple’s does around personal intelligence. But you can also connect Google services to all kinds of things: your work and personal email, your search history, your Chrome history. There are a lot more surfaces. Each one may have a different policy. If you have a Pixel phone, for example, Pixel phones or Nest hardware have different privacy policies than perhaps the Google service you might be using on that device. So that one’s a more complicated one. I understand why they have a million policies, but it doesn’t make life easier for the person who’s trying to understand, how is my data being used?

EL KALIOUBY: Yeah. What about Anthropic?

FRIED: Anthropic’s interesting. They are not as consumer-focused, so Claude, despite having a very friendly name, is mostly used by businesses, and that remains true. They have taken one particularly interesting stand on the consumer side by saying, “We’re not going to have advertising. We’re not doing it today, and we’re not going to do it.” And that, to me, is more significant than it initially sounds because it does keep them closer to this idea that your AI assistant should be working for you. And I do think OpenAI, to the degree advertising becomes a significant part of its consumer business, is going to struggle with those divided incentives. And again, it’s not just, are they trying to persuade me to buy this particular thing, but they now have an incentive to keep me using it longer. I worry that, especially with teens and kids, with people with underlying mental health issues, the potential for addiction, the potential for unhealthy companionship — because I think there is the possibility of healthy companionship with AI. But returning to Anthropic, I do think not having advertising is a big deal.

EL KALIOUBY: Although things can change, right?

FRIED: They can. I mean, they’ve said, “We’re not doing it, period,” but that’s not legally binding. And that’s true of all of them. And I do think, again, that’s why I imagine, when I’m sharing information with a chatbot, how might this be used in the future? Because privacy policies change, CEOs change, economic necessities change. And the other one we’ve seen that’s really scary is companies go out of business, and that’s a tricky one across the board. Google is trying to buy Spirit Airlines’ data. I saw this with an AI companion toy that went out of business. It was problematic for a different reason. Kids had these connections to this AI toy, and then overnight it stopped working. Again, there’s only so much any of us as consumers can keep in mind, but what if this company went out of business? What if that data were sold to the highest bidder? That is maybe on the list, at least, in a concerned person’s mind.

Copy LinkWhy AI privacy risks feel more urgent now

EL KALIOUBY: Yeah, absolutely. So this series is a continuation of a project that you had done in 2019. Why the urgency? Why now?

FRIED: Yeah. I really see a world in which what companies can do with that data, separate from how they train their systems, is going to be vast. And I can imagine an AI bot that’s built to keep me online all the time. I can imagine an AI bot that’s built to persuade me, whether that’s to persuade me to buy cosmetics or weight-loss supplements, whether that’s to persuade me to buy more of a particular product than I might need, or literally change my opinion about something consequential. That’s the world I imagine we are headed into. And all we have to go on, because the legislation has been so slow, is what the companies say. And there’s some nibbling at the edges. The EU, as usual, is a little bit ahead. So they have some limits on what companies can do.

EL KALIOUBY: But even the AI Act, I feel like, is already outdated. I mean, I haven’t looked to see if they’ve made any recent revisions, but I feel like the pace of innovation is so much faster than some of this legislation.

FRIED: Yeah. And again, I think that’s going to continue. I think it’s almost impossible for any of us to keep up with this pace. And now the computers are going to start training themselves, so it’s only going to get faster.

EL KALIOUBY: So I think my big takeaway from this conversation is that it is so complex for the average consumer to figure out what data is being collected and how it is being used or could be used. So what’s your advice to our listeners on how to wrap their heads around this?

FRIED: The goal of this series is to help people so they have a little bit less to do. So that’ll be online at axios.com. But I think picking probably a primary service that you’re comfortable with, based on its privacy policy, and being a little bit active in your thinking — what am I sharing, and am I OK with this kind of being there forever? And if I’m not, do I want to take the added work? And it is work to say, forget about this. Don’t keep this in memory. Or do I use an incognito chat of some kind? Those things are not unreasonable steps. And I get it. I’m super busy. Also, when you’re using consumer services that have the most expansive privacy policies, it’s worth thinking about how much data we’re sharing with those companies. Again, some of them provide a lot of utility, and I’m not saying you shouldn’t use any of these companies. But I do think, for as much and as deep as some of the stuff that’s being shared is, it’s worth a thought: Where is that data going?

EL KALIOUBY: Ina, thank you so much for joining us on the show. This was really helpful and fascinating.

FRIED: Thank you, Rana. And thank you for having these important conversations. This technology is amazing. It is going to change our lives. But understanding how, and really creating a voice for people to have a say — because we do have a collective ability to say this is OK with us and this is a bridge too far — I think it’s worth remembering that we do have the power of the purse.