{"id":1104,"date":"2019-08-31T16:59:20","date_gmt":"2019-09-01T00:59:20","guid":{"rendered":"https:\/\/accretiondisc.com\/blog\/?p=1104"},"modified":"2019-08-31T17:26:39","modified_gmt":"2019-09-01T01:26:39","slug":"duplicate-files","status":"publish","type":"post","link":"https:\/\/accretiondisc.com\/blog\/2019\/08\/31\/duplicate-files\/","title":{"rendered":"Duplicate Files"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">I&#8217;ve been hoarding data for more than 20 years. For backups, I used to burn a CD periodically, but I long since ran over those limits. Today, my backups are hard drives. One reason is that I&#8217;ve moved between computers several times during that period, and when I do, I find stuff I don&#8217;t know what to do with. So I copy all that data into a new folder, typically called something like <em>temp\/backup\/that-system-name\/tmp\/old\/save\/keep\/t.files\/save.d<\/em>. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">After 20 years, that starts to add up. So I&#8217;ve been looking at programs to help me find and get rid of duplicates. (I&#8217;ve been using <em>rsync -n<\/em>, and occasionally <em>diff -qr<\/em>, to compare folders. But the problem is deciding what folders, at what places in the directory structure, to compare.)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">So I&#8217;ve been looking to see what kind of tools are available to help. At this point, I looked at <a href=\"http:\/\/duff.dreda.org\/\">duff<\/a>, <a href=\"https:\/\/github.com\/jbruchon\/jdupes\">jdupes<\/a>, and <a href=\"https:\/\/github.com\/jvirkki\/dupd\">dupd<\/a>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">So far, I&#8217;ve focused on dupd. It does what I was thinking needed to be done: crawl the entire hierarchy and save the result as a database.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I&#8217;ve been hoarding data for more than 20 years. For backups, I used to burn a CD periodically, but I long since ran over those limits. Today, my backups are hard drives. One reason is that I&#8217;ve moved between computers several times during that period, and when I do, I find stuff I don&#8217;t know [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[52],"tags":[646,1036,1035,809],"class_list":["post-1104","post","type-post","status-publish","format-standard","hentry","category-technology","tag-backup","tag-comparison","tag-data","tag-system-admin"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/paRqpr-hO","_links":{"self":[{"href":"https:\/\/accretiondisc.com\/blog\/wp-json\/wp\/v2\/posts\/1104","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/accretiondisc.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/accretiondisc.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/accretiondisc.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/accretiondisc.com\/blog\/wp-json\/wp\/v2\/comments?post=1104"}],"version-history":[{"count":0,"href":"https:\/\/accretiondisc.com\/blog\/wp-json\/wp\/v2\/posts\/1104\/revisions"}],"wp:attachment":[{"href":"https:\/\/accretiondisc.com\/blog\/wp-json\/wp\/v2\/media?parent=1104"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/accretiondisc.com\/blog\/wp-json\/wp\/v2\/categories?post=1104"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/accretiondisc.com\/blog\/wp-json\/wp\/v2\/tags?post=1104"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}