Getting a Tarball Off Android Doing Newline Substitution

I had a problem, my attempts to output a tar file to STDOUT and onto my local filesystem using adb were bearing no fruit, instead getting the error:

tar: Skipping to next header
tar: Exiting with failure status due to previous errors

I was using the command:

$ adb shell "tar -cf - /sdcard 2>/dev/null" >mycopy.tar

Later I discovered that all instances of character 0x0A (\n) were being replaced with 0x0D 0x0A. and this was corrupting the output file.

The work-around was to use sed:

$ adb shell "tar -cf - /sdcard 2>/dev/null |sed 's/\n/_WOW_A~NEW~LINE~~/g'" \
    |sed 's/_WOW_A~NEW~LINE~~/\n/g' \
    >mycopy.tar

This converts all \n characters to a long complex string we’re fairly confident won’t ever appear in the data before delivering across adb. Then converts that long complex string back into \n characters before saving to the file.

To extract a tarball the following command can be used:

$ cat data-backup.tar.gz \
  |sed 's/\n/A~NEW~LINE~~/g' \
  |adb shell "sed 's/A~NEW~LINE~~/\n/g' |tar -xvzf - -C /path/to/extract/to"