Question about a bug encountered while transferring photo and video files between devices
This is my first Tildes post and I'll remove it if needed!
I recently dumped some photos from an old cell phone on to an old windows 10 laptop to be stored on an external hard drive.
The phone is a 4 year old Galaxy with 128g onboard storage.
The laptop is an HP running windows 10 and is a notebook-like machine with about 30g total hard drive, the max usable is like 4 or 5 gigs after the OS etc.
At the time of transferring files, I found it quicker to use the available 2.5 gigs I had to put pictures directly on the laptop and then transfer them from there to the external hard drive.
Here is my problem:
2 folders, from separate camping trips, totalling about 380 photos and a few videos are stuck on the desktop and are claiming to take up 4.02 terabytes and thus cannot be moved.
I did notice the file type .heic is not recognized by windows 10, but all my other photos (several thousand,) are the same file type and take up a normal amount of space.
These individual photos in question are claiming to be around 7 to 8 gigs each.
There's not 4 TB between the phone(128g,) laptop(30g,) and the external drive(3tb.)
So the pictures are stuck on this laptop which is only acting as a surrogate computer while I'm building a real desktop PC.
I can keep this laptop forever, even though I'd rather donate it or something, but one of these folders has pictures from the last camping trip with my brother before he took his own life last year, I'd really like to keep them archived and backed up.
Any ideas? Anyone have a similar experience? Thank you for reading and thanks in advance for any suggestions!
Again, I'll delete this post if it's inappropriate.
Cheers.
EDIT: I just realized while proof reading this, that if I can update the codecs where windows can view the files, I could screenshot the photos, but that still leaves me at a loss for the videos.
I miss his goofy laugh, and want to preserve it for his son also.
If I'm understanding this correctly, Windows is not accurately portraying file sizes / thinks files are bigger than they are? My initial thought it to cut Windows out of the mix and boot from a Linux live USB, mount the Windows NTFS partition and take a gander at the photos from the Linux OS. If the file size is still abnormally large, then something went wrong with a transfer from the phone to the laptop and the issue is with the NTFS partition (if I understand the chain of events correctly), if the file sizes are normal, it's the Windows install. If the files look OK from the Linux USB, I'd proceed to mount the external USB hard drive within the live environment and copy them from the laptop to the HDD.
To play it safe, since these are important files and I always gets nervous when windows starts having file system issues, let me ask: are the photos and videos still on the phone? If the originals are still on the phone, it might be prudent to buy a USB OTG dongle and a flash drive to move the files directly from the phone to another storage medium like the external HDD or a USB drive. Cut the laptop out of the picture.
Unsure on the exact root cause, but interested in your findings if you choose to live boot Linux.
Edit: Also, welcome to Tildes!
based on some Microsoft forum posts, this does indeed look like a Windows bug:
https://answers.microsoft.com/en-us/windows/forum/all/windows-10-showing-the-wrong-file-size-in-version/8e4b1d9c-53a6-4767-b97f-70a9d4a0a4cf
https://answers.microsoft.com/en-us/windows/forum/all/size-on-disk-much-larger-than-actual-size-how-to/9428b528-8326-45fb-a056-21815b11b984
seems incredibly stupid to me that they're parsing the JPEG file metadata to determine its size rather than using NTFS's own metadata, but Windows gonna Windows.
@mr-death, I think the Linux live USB suggestion is probably the best way to go here.
here's a how-to on getting Ubuntu installed to a USB stick if you need it: Create a bootable USB stick with Rufus on Windows
and then you'll need to mount both the external drive and the laptop's boot drive from within the Ubuntu live environment. the quality of how-tos for doing this is all over the place, this one looks reasonable.
I'd definitely recommend mounting the laptop's boot drive in read-only mode (
roin the options string) to avoid any risk of inadvertently deleting or corrupting the only copy of these photos & videos. the external drive should be mounted in read-write mode, and then you can copy the files over to it.an alternate option would be to apply any Windows updates that are pending, because there may be an update that fixes this bug with HEIC files. I'm hesitant to recommend that though, because of the risk that the updates might corrupt something (especially with so little free space on the disk already) and make it even harder to recover the files you care about.
Thanks for doing the requisite research! Not shocked at all by this bug. Great points about read-only! I'd be very hesitant to engage in Windows updates at this point. I have no idea how Windows update will handle potentially running out of space, as Windows 10 updates can take gigabytes.
Unfortunately, I removed them from the phone after moving them to the PC, but before I realized the problem. I transferred around 10,000 files, but only these 2 folders, 478 pictures and videos had this issue. I normally never allow thousands of photos to accumulate on my phone, but it's been a strange year.
I wonder how small a Linux partition could be these days? Even with these 478 files claiming to be 4.02 Terrabytes, my laptop hard drive says it has 2.81 gigs free of 27.8. I could perhaps make the available space larger, but maybe not.
Non of it really makes sense to me.
One thing you could do, is if you have free space on the external HDD is (using Windows) download the Linux ISO onto the external, and then burn it to a separate USB drive. If that's not an option for some reason, WinDirStat, https://windirstat.net/, is what I've used to clean out unneeded files. It gives a visual representation of what's taking up the most space. I strongly encourage you to only remove files from your user directory you know you don't need like old Downloads. I wouldn't venture into system/temp files unless you're confident you know what those do, and that you don't need them. Windows 10 also has a tool to free up space if you go to the Start Menu and type "Delete temporary files." This will remove system/temp files safely.
I am siding with the other suggestions to use a live Linux boot disk to recover these, but please, please do not do this if you have never used a live Linux boot disk before or are otherwise uncomfortable with it.
They make it kind of hard to accidentally overwrite data with lots of warning messages, but it is still just a few prompts. You can't get the data back if you do overwrite it.
There are surely plenty of local people that would be more than willing to help you make sure that the videos are not lost (myself included, if you live somewhere within a couple of hour drive of me).
Thank you so much! I live in northeastern Oklahoma currently.
I haven't done anything with Linux since repurposing an old Pentium (3 or 4,) windows 98 machine in 2007 to run Ubuntu, so I could record music and better surf the web.
I used to know my way around pcs better (circa 1999 to 2010ish,) but I'm out of practice and out of the loop on most stuff these days. I'm building a PC now, have all the parts, but I'm afraid to mess it up without supervision, ha
I can watch some videos and read some tutorials regarding the Ubuntu drive until I feel better about it, but it'll just take some time.
I don't know what else to do, the whole thing is just strange.
Oklahoma City has a Linux User Group with a virtual meeting this Friday, you may be able to find someone local who can help you out. Tulsa apparently used to have a LUG too but it seems to be inactive (or maybe subsumed into the OKC group).
you could possibly just point them at this thread, but if you want to give them a simplified explanation of your problem:
(in general, if you tell Linux users you're trying to work around a stupid bug in Windows, they'll immediately understand)
a local computer repair shop might be able to help you as well, but they would probably either a) misinterpret what you're asking for as data recovery from a failed drive and want to charge you $$$$$, or b) not understand that copying the files under Windows won't work and it needs to be done running Linux.
Install Oracle VirtualBox, create a virtual machine, install Ubuntu on it. Then you can put in practice the advice the other people gave you, except on a virtual machine that you can mess up by accident, but without consequence.
Also, the modern Linux distros are easy to install. For Ubuntu it's mostly clicking next next next on some GUI.
That's great for general linux practice, but won't really help a lot in this situation. The main thing we're trying to do is copy files off one drive onto another, but virtualbox won't be able to mount your windows drive since it's in use, and won't be able to mount the USB drive without first passing it through to the VM.
Also, they only have 2gb of free space, so that's not a lot of room for a linux VM.
HEIC is a newer image type that IIRC uses the same techniques as HEVC. It provides higher quality images. JPEG is quite old now so if you were to transcode them to that format they would likely take up even more space unless you shrink the resolution or use a lower quality setting.
I dont know why it’s showing the files as being so large, so I don’t have a solution for you there, but if your computer has so little storage you should get an external storage drive to hold your files. Full disks make your computer run slower, even if it is an SSD.
I do have multiple external drives, the laptop was just a temporary placeholder for what I figured would be a few hundred megabytes at best, but now at 4.02 Terrabytes for less than 500 files they are stuck.
I make no promises that this will work because it has been a while since I've had to do it, but here goes.
Go here and input this link to the HEVC video extension.
Download and install the .appxbundle it returns then restart the computer. There's also the image extension you can get the same way if needed.
I've had success with this method before so it's worth a shot, but as for why the files are saying they're multiple gigs I really couldn't answer. Potentially file corruption? I hope you can get it figured out.
Thank you, friend! I'll look into trying this and I'll report back.
Cheers!
I just thought of it and wanted to come back to add that VLC might be able to help you out with the screwed up file size assuming it's just a problem reading the heic header like someone else suggested.
VLC should be able to open the video and convert it out to a more standard format that potentially wouldn't have the file size problem. I know you're getting a lot of different advice here, but I wanted to add this as another option just in case.
Good luck friend.
If you can afford it this is honestly a good use case for buying a google one subscription with google photos (or same with google workspace) and transferring everything on there, letting it deal with quality etc.
Few bucks, huge time saver.
Yes, it is odd that it claims to take 4 terabytes if the laptop doesn't have that much space. I don't know the answer, but here are some things it might be:
Whatever you're looking at might be reporting the amount of space the files would take up if uncompressed. Possibly the entire disk drive is compressed? I don't know if that's a thing on Windows.
Some operating systems have a concept of a "sparse file" where some space is reserved but not used. It seems unlikely for a photo or video, though.
I think the first step is to narrow down the problem. Which files are suspiciously large? I would guess it's the videos?
One alternative is to buy another disk, if you're not concerned about what it would cost.
If it's the videos, you could also find some software that understands the video format and can convert them into a different format, perhaps editing them down. Maybe you don't need full resolution video files? If it's a long, unedited video, maybe there are parts of the videos that are "dead space" and could be edited out?
The software I would use for this is called Handbrake, which is free and open source. It's a compression and conversion tool, not a video editor, though. It lets you preview what the converted file would look like, so you can make a choice about resolution.
It may be tricky to use if you don't have any free space left for the new file, though.
Thank you! I will look into this option. One folder has 90 files, 13 videos and 77 photos. It is showing each photo at 14.1 gigs and each video at 14.2 gigs.
The other folder has 388 files, 140 are pictures showing 7.32 gigs each. And the remaining 248 video are all showing 7.38 gigs each, even though every video is a different length.
These files all came through a USBC cord from my phone to my laptop. All of these files were taken with the same camera, but range from MP4, jpeg to heic files.
I have to wonder if this is a problem only because you are using the GUI explorer to copy files around. I would expect that if you just went into the command shell or whatever, this would simply work.
I would try instructing 7-zip to create an uncompressed i.e. "store" (they're already highly compressed, so it's faster) archive of them in the target location.
If it's a HEIC metadata bug, it won't care. If it's a symlink issue, it will dereference them. It's as effortless as it gets, and already solves the "archiving" goal you've set.
Or, use IrfanView batch mode to make webp lossless files, since these appear corrupt.
I think 7zip creates a temporary archive on the system drive before copying it to the destination. If we're down to 2gb free space this might be an issue.
Just tested between two physical drives. Archiving 500 GB C:\test\ to D:\ with 50 GB free. 0-byte D:\test.7z.tmp is created and then progressively appended to from memory. Raises no issues with free space, of course, since it can't predict the end file size.
You could do it one file at a time, archive + copy + delete archive, probably pretty easy to do with a batch file.