13 votes

TSA Facial Recognition is here, and it works distressingly well.

My wife and I recently spent 10 days in Ireland. On our trip home, we went through US Customs at the Dublin airport. When it was our turn to speak to the TSA agent, we were called over to the desk and the agent asked my wife to stand in front of the camera. I assumed she was just getting her picture taken, similar to what we had done by Jamaica Customs when we flew into Montego Bay a couple years ago. Before she could even give him her passport or boarding pass , the agent asked "So <wife's name>, what part of Wisconsin is <city we live in> located?". As soon as it happened, my stomach dropped. I've been thinking about it ever since and still can't quite figure out how they are populating their facial recognition database. A couple more points:

· There was nothing else the TSA agent could have used to identify my wife.
· We had not had pictures/facial scans taken in any US Federal facility prior to this.
· From the time my wife stood in front of the camera to the TSA agent starting to ask, maybe 3-5 seconds had elapsed.

Has anyone else experienced this? This Twitter thread with JetBlue seems to be a similar experience.

9 comments

  1. [4]
    bbvnvlt
    Link
    Do they not have the pictures from passports on file or something?

    can't quite figure out how they are populating their facial recognition database

    Do they not have the pictures from passports on file or something?

    7 votes
    1. [3]
      seizethegoddamngap
      Link Parent
      That's probably it, but it seems like matching a face in 3-5 seconds off of a single 1.5" picture would be highly improbable. Maybe the database is constantly updated with only people who are...

      That's probably it, but it seems like matching a face in 3-5 seconds off of a single 1.5" picture would be highly improbable.

      Maybe the database is constantly updated with only people who are currently in country which narrows the results and greatly reduces false positives?

      1 vote
      1. Greg
        Link Parent
        The database is likely pre-populated from the passport data, and many of the rules about how passport photos are taken are specifically designed so that they can get a good facial biometric from...

        The database is likely pre-populated from the passport data, and many of the rules about how passport photos are taken are specifically designed so that they can get a good facial biometric from the image. It's been fairly standard for the last few years to go through fully automated customs & immigration in some countries: touch the passport to the reader, stand in front of the camera, and that's it.

        Of course, when they read the passport chip first it means that the realtime photo can just be verified against that one person's data. I'm not particularly familiar with how facial recognition works mathematically, but I can say that a general lookup across 300m records of well indexed data isn't particularly onerous. Given that identifying an individual within a large population is a fairly common application of the technology, I would make an educated guess that researchers have put a lot of time into ensuring that the data can be stored and indexed in a way that makes this efficient.

        In terms of reducing the search space (if necessary), I believe that the airlines are required to submit passenger manifests for all international flights entering the USA - I imagine a well designed lookup system would pull the data from those manifests and push it to the top of the facial recognition stack.

        9 votes
      2. bbvnvlt
        Link Parent
        If the images are converted into a set of numerical characteristics (sizes/proportions) then a database lookup among 300 million records in 3-5 seconds doesn't seem that outrageous. I mean, Google...

        If the images are converted into a set of numerical characteristics (sizes/proportions) then a database lookup among 300 million records in 3-5 seconds doesn't seem that outrageous. I mean, Google returns "About 237.000.000 results (0,62 seconds)" as a response to "facial recognition"...

        3 votes
  2. [5]
    Keegan
    Link
    It hasn't worked on me the two times I have tried it. I suppose that could be due to me raising my eyebrows and putting my tongue in my upper lip to change my appearance slightly, but idk.

    It hasn't worked on me the two times I have tried it. I suppose that could be due to me raising my eyebrows and putting my tongue in my upper lip to change my appearance slightly, but idk.

    1 vote
    1. [4]
      cfabbro
      Link Parent
      I thought most facial recognition models use reference points which are not modified with facial muscle or jaw movement (e.g. distance between eyes, cheekbones, nose bridge, brow ridges, temples,...

      I thought most facial recognition models use reference points which are not modified with facial muscle or jaw movement (e.g. distance between eyes, cheekbones, nose bridge, brow ridges, temples, etc), so contorting your face won't do you much good?

      6 votes
      1. [3]
        Keegan
        Link Parent
        I have no idea how it works. It worked on my whole family the past two times, but not for me. Maybe they just lie and say it didn't work a few times to get people thinking it isn't as developed as...

        I have no idea how it works. It worked on my whole family the past two times, but not for me. Maybe they just lie and say it didn't work a few times to get people thinking it isn't as developed as it actually is.

        2 votes
        1. [2]
          Greg
          Link Parent
          Do you wear glasses, by any chance? The automated machines didn't used to explicitly say to take them off, and I've had a 100% failure rate when wearing them but a near-100% success rate when I...

          Do you wear glasses, by any chance? The automated machines didn't used to explicitly say to take them off, and I've had a 100% failure rate when wearing them but a near-100% success rate when I remove them.

          1. Keegan
            Link Parent
            I do actually. That might explain it then haha.

            I do actually. That might explain it then haha.