9 votes

How Lucasfilm’s new “stagecraft” tech brought ‘The Mandalorian’ to life and may change the future of TV

5 comments

  1. [5]
    balooga
    Link
    Having read that, and ILM's official marketing page for StageCraft... I still have no idea what it is.

    Having read that, and ILM's official marketing page for StageCraft... I still have no idea what it is.

    2 votes
    1. [4]
      Akir
      Link Parent
      It's basically a new way to do virtual sets. The same source has a different article with a video to show what's happening.

      It's basically a new way to do virtual sets. The same source has a different article with a video to show what's happening.

      6 votes
      1. [3]
        balooga
        Link Parent
        That's definitely more helpful, thanks! But I still have a lot of questions, and I guess there's a big effort being made here to safeguard some trade secrets. What I want to know is, is the camera...

        That's definitely more helpful, thanks! But I still have a lot of questions, and I guess there's a big effort being made here to safeguard some trade secrets.

        What I want to know is, is the camera recording the image from the screen behind the actors as if it were a physical background, and not compositing the background in digitally? I assume not, but I'm not sure how the compositing is being done at high-enough quality without a solid color background to chroma key, or some sort of digital tool that would typically not be quick enough to work in realtime. I suppose if the system knows exactly what image is being displayed on-set in a given frame, it would be possible to compare that with the in-camera image and extract the foreground from it that way.

        The video shows realistic reflections on the motorcycle and actor's sunglass lenses. But the screen is behind him, not in front of him. It's not clear where those reflections are coming from, or if there's some post-production trickery happening here.

        The image on the screen appears to be paired with the motion of the camera, so as the camera pans the background displays a convincing parallax. Looks good in-camera but it seems like that would be fairly disorienting for the actors.

        2 votes
        1. WendigoTulpa
          Link Parent
          I don't know how they did it at ILM, but they could have used a 360 degree camera to capture a panorama that is synced to the camera position to have the footage move with the camera. Seems like...

          I don't know how they did it at ILM, but they could have used a 360 degree camera to capture a panorama that is synced to the camera position to have the footage move with the camera.

          Seems like in the video that was linked they used the footage and added some real-time assets to be composited into the shot and rendered real time as the camera moved.

          Since the background is a screen, you could even use a cross-polarization technique to create a mask that separates the real-life scene from the screen, thus allowing you to composite even more in after the fact.

          Again, just conjecture on my part, I have no idea but I would be surprised if they weren't using 360 panoramic cameras to get the location footage.

          EDIT: No clue how they did the reflections. Maybe post-processing.

          2 votes
        2. Akir
          Link Parent
          It seems like the film industry has always had a "semi-open" development strategy that keeps the broad strokes open so that filmmakers can figure out how the technology helps them without giving...

          It seems like the film industry has always had a "semi-open" development strategy that keeps the broad strokes open so that filmmakers can figure out how the technology helps them without giving the fine details of implementation. A cameraman, for instance, will know how to record a scene, but that doesn't mean that they know any of the steps that it takes to turn the incoming light into digital data.

          For this in particular, I think that they have a system that is currently very proprietary but if you were to see all the details their competition could figure out how much of it works. It's an especially big deal if they are using trade secrets instead of patents.

          1 vote