![]() |
|
Peer to Peer The 3rd millenium technology! |
![]() |
|
Thread Tools | Search this Thread | Display Modes |
![]() |
#1 |
Join Date: May 2001
Location: New England
Posts: 10,023
|
![]() Since 2002
![]() October 24th, 2015 ![]() Pirate Bay's Internet Provider Is Getting Sued for the Site's File-Sharing Bredbandsbolaget has allowed the Pirate Bay to exist on its servers. Is it liable for piracy? Matthew Strauss A Stockholm District Court will decide whether torrenting website the Pirate Bay should be blocked in Sweden, the country in which it was founded. The case stems from a November 2014 lawsuit filed by Universal Music, Sony Music, Warner Music, Nordisk Film, and the Swedish Film Industry against Swedish internet service provider (ISP) Bredbandsbolaget, the broadband company that alleges the ISP knowingly violated copyright law by allowing subscribers to access the Pirate Bay. Following the suit, Bredbandsbolaget explicitly stated in December 2014 that it would not block file-sharing, copyright-infringing services. According to Torrentfreak, the company said, “Bredbandsbolaget’s role is to provide its subscribers with access to the Internet, thereby contributing to the free flow of information and the ability for people to reach each other and communicate.” The statement continued, “Bredbandsbolaget does not block content or services based on individual organizations’ requests. There is no legal obligation for operators to block either The Pirate Bay or Swefilmer.” This is the first time the big entertainment companies are targeting an internet provider, TorrentFreak reports. Previously, sites like the Pirate Bay itself were the main violators, but December’s statements suggest a level of accountability to which Bredbandsbolaget may be held. The case is particularly significant because if the ISP is found liable, it could open up more avenues for lawsuits and shutdowns of servers that allow illegal websites to operate. Granted, Bredbandsbolaget publicly declared that it cannot be held responsible for what its customers put online. It’s entirely possible that servers cannot track every shady website it allows to exist. ISPs would have to dedicate a lot of new resources to maintaining and offering a clean slate of websites for users. As Bredbandsbolaget spokesman Aron Samuelsson said, “It is dangerous if we are sentenced as accomplices or participants to crimes committed online. In this case it is about copyright infringement, but it is difficult to logically explain why it would stop at that.” For internet users who like to download free movies and albums and books, this is worth following. https://www.inverse.com/article/7354...s-file-sharing ![]() ‘Netflix for Pirates’ Now Streams Movies Directly to Your Web Browser Zach Epstein An app released earlier this year called Popcorn Time caused quite a stir. Whereas downloading stolen movies with torrent applications had previously been something that “mainstream” users stayed away from, this “Netflix for pirates” app made stealing copyrighted movies and TV shows as easy as streaming a video on Netflix. The app provided a beautiful interface that rethought the multi-step, multi-app torrenting process and combined everything into one simple app. Thanks to a new project spawned by Popcorn Time, the process has been further simplified and you don’t even need to install an app anymore. Local apps have taken a back seat to web apps on desktop platforms in recent years. Rather than cluttering up a PC or Mac, people often prefer to simply bookmark a website that can accomplish nearly the same thing. This might be especially true for an app like Popcorn Time, which isn’t exactly legal considering it facilitates theft. Enter Browser Popcorn, which offers users an experience akin to Popcorn Time but exists solely in a web browser. Rather than having to download an app, Browser Popcorn is a simple website that lays out available movies and shows in an interface similar to Popcorn Time, which of course is modeled after Netflix. Gone are the days of using dedicated torrent search sites to find files, a separate client to download them, and then yet another app to watch. With Browser Popcorn, simply select a title and it will begin streaming directly to your browser. The site lists its most popular titles directly on the homepage, and then users can drill down by categories such as drama, action and sci-fi. There’s also a search mechanism, of course, which allows users to search for specific titles. https://bgr.com/2015/10/19/popcorn-t...d-browser-app/ ![]() ‘The Song Machine,’ by John Seabrook Touré In my 20s I wrote about music for Rolling Stone magazine, which put me in contact with all sorts of record business people — artists, hustlers, machers, you know the cast. Many of them were aggressive and arrogant and flaunting their success at Mr. Chow or in St. Barts. It was the ’90s, and as an industry the music biz was doing better than it ever had. In 1999 worldwide revenue was $27 billion, and it looked like nothing but blue sky ahead. John Seabrook’s well-researched new book about the industry covers the sharp downward slope that followed. The people I knew combed the country looking for the next hit makers and chatted with global conglomerates about acquisitions and weekended in the Hamptons, with no idea that the end of the party was right around the corner. That their industry was going to contract so much that in 2014 worldwide revenue would be $15 billion, not much more than half of what it had been at its peak. There’s still a music business, but it’s a lot less profitable and a lot less arrogant than it used to be. Much of the fault has been laid at the door of Napster and the digital Pandora’s box it opened. It’s true that piracy did significant damage to the music business model — if suddenly people can get your product free, it’s going to be hard to sell it. Steve Jobs helped the industry put the genie back in the bottle a little, but iTunes’ success with selling songs has been a Pyrrhic victory: An industry that was used to selling albums for around $15 was now peddling songs for 99 cents — which, according to Seabrook, caused a 46 percent drop in revenue. But there have been other unfortunate changes that also hurt the business. The music biz used to be a herd of labels, each with its own character, some of them run by people who were said to “have ears.” These were people who could hear a hit, or even better, who could hear a consistent hit maker, before anyone else. They were the high priests of this world, people like Clive Davis, Tommy Mottola and Ahmet Ertegun. They understood how to manage creative people, and if artists needed more time or money before finishing their albums, they got it. But the industry’s success led to labels being snapped up by global conglomerates, which has led to music companies sometimes being run by people who are, let’s say, less sensitive to the needs of creative people and more focused on quarterly reports. Also, as record sales dropped and less money could be made from recording, artists began touring more relentlessly. But albums are still helpful for increasing ticket sales, so albums must still be recorded. That means lyrics are sometimes written on a tour bus as it moves from city to city, and vocals are recorded in makeshift studios in hotel rooms. This seems like a harder path to making meaningful music than settling into a beautiful studio in Los Angeles or Memphis or King#ston and staying until the spirits deliver a heavenly gift. And all of that is why over the last dec#ade we have watched the music business slowly atrophy. It’s withering away before our eyes. Not only are the people inside it no longer so arrogant; in some cases they’re pretty anxious. They’re watching the last gasps of a dying model. Or, as Thom Yorke called Spotify, “the last desperate fart of a dying corpse.” In “The Song Machine,” Seabrook — a staff writer at The New Yorker and the author of “Nobrow: The Culture of Marketing, the Marketing of Culture” — takes us inside the troubled modern music business. We go behind the curtain to meet some of the producers, executives, songwriters and artists responsible for the hits that our kids listen to on Spot#ify, songs that, to Seabrook, seem like #“industrial-strength products, made for malls, stadiums, airports, casinos, gyms and the Super Bowl halftime show. The music reminded me a little of the bubble-gum pop of my preteen years, but it was vodka-flavored and laced with MDMA.” (MDMA is a sweet flavor compared with the drug that inspired a recent No. 1 song on the Billboard chart: “Can’t Feel My Face” by the Weeknd — say “Weekend,” Mom — which is about the artist’s relationship with cocaine. But I digress.) These songs/products are quite often a sort of three-minute advertisement for how powerful or sexual or beautiful or awesome the artist is. “The artists occupy a central place in the songs,” Seabrook writes, “but more as vocal personalities than singers. . . . What do they stand for as artists? Their insights into the human condition seem to extend no further than the walls of the vocal booth.” It does seem as if the political spine that was provided by the likes of Bob Dylan and Public Enemy and Radiohead has grown soft. But no matter — what the music biz cares about most is making hits. One of my exec friends once told me, “One hit pays for 10 flops.” O.K., but what happens when the hits become rare and the ones you find don’t become as big? The struggle to sell truckloads has only increased the frantic search for monster hits. Seabrook takes us through the world of people who are trying to make those hits. He hangs out at a writer camp where an array of songwriters lay up in a hotel for a few days, banging out multiple songs a day, trying to create a hit for Rihanna. He introduces us to top-liners — people who write catchy, hummable lyrics and sing the demo that lets a star know what to sing and how to sing it. He shows us the tension between artists and executives, as when Kelly Clarkson records a song but later tells Clive Davis, then the chairman and chief executive of RCA Music Group, that she hates it. Davis basically says too bad and regales her with stories like the one about Barry Manilow, who hated “I Write the Songs” and initially refused to record it. It went to No. 1 on the Billboard Hot 100 and won a Grammy for song of the year. Hearing this, Clarkson digs in her heels. She cries, according to Davis’s account in his memoir. The rec#ord man doesn’t budge. The song they are arguing over is “Since U Been Gone,” which reached No. 2 on the Billboard Hot 100 and sold over a million copies. Don’t argue with Davis’s ears. Seabrook also takes us to a Swedish producer’s hit-making factory. I hadn’t fully grasped the large impact that Swedish producers have had on modern pop music and the reasons there are so many great producers and songwriters in that nation. “Swedes are very musical, and they love to write songs,” says Klas Ahlund, a Swedish songwriter and producer. “Songwriting was just a thing you did on your own when you were watching the cows, a kind of meditation. You didn’t focus as much on your ability as a performer as you did on the structure and craft of the songs. Which is really not the case in the U.S., where your charm and your voice and your powers as a performer come immediately into play.” For most of the book, Seabrook describes a machine that appears to be working fine. We’re just out here trying to make hits. What dying model? There are no problems to see here! Move along! And don’t forget to dance while you do! But toward the end we meet Daniel Ek, the founder of Spotify, and then we start to see the unraveling. Spotify has picked up where iTunes left off, furthering the erosion of revenue — why buy one album for $15 when you can rent the entire library for $9.99? And artists’ income from streaming is a small fraction of what it was from sales. This is partly because there’s less money in streaming, but there’s also this: “Month by month, Spot#ify pays the major labels lump sums for the entire market share of their catalogs. How the labels decide to parcel these payments out to their artists isn’t transparent, because, while Spotify gives detailed data to the labels, the labels ultimately decide how to share that information with their artists.” Seabrook quotes an unnamed industry leader: “It’s like you go to your bank, and the bank says, ‘Here’s your salary,’ and you say, ‘But what is my employer paying me? I work for them, not you!’ And the bank says, ‘We are not going to tell you, but this is what we think you should get paid.’ ” Clearly this system is privileging the stars, whom labels need most, over the lesser lights. Seabrook speaks to Rosanne Cash, who said she made $104 from 600,000 streams. A lot of artists are getting less of a cut of a shrinking pie. But what’s happening to songwriters is much scarier, and it has the potential to truly kill the industry. In order to get into business in America, Spotify struck a deal with the labels that does not give much to songwriters: The owner of the recording, the label, gets most of the money, while the owner of the publishing rights, the songwriter, gets a teeny piece. “If streaming is the future,” the songwriter Savan Kotecha says, “no young songwriter will be able to make a living.” This whole house of cards called the music biz is built on songwriters coming up with hits that a superstar can sing. What happens without them? “If songwriters can’t afford to work,” Seabrook writes, “then the whole hit-making apparatus of the song machine is doomed.” THE SONG MACHINE Inside the Hit Factory By John Seabrook 338 pp. W.W. Norton & Company. $26.95. ________ Touré is the author of several books, including “I Would Die 4 U: Why Prince Became an Icon.” http://www.nytimes.com/2015/10/18/bo...-seabrook.html ![]() Independent Musicians Find Unexpected Rewards in Streaming Ben Sisario Early last year, Perrin Lamb, a singer-songwriter in Nashville who is not signed to a record label, started to receive all kinds of strange Twitter messages. Fans he never knew he had, writing sometimes in languages he couldn’t understand, were saying that they loved his song “Everyone’s Got Something” on Spotify. Mr. Lamb — who did not use Spotify — quickly learned that “Everyone’s Got Something” was on a popular playlist on the service, and racking up streams by the million. By the end of the year, the song had been listened to some 10 million times, earning Mr. Lamb more than $40,000. “Whoa,” he recalled thinking. “I should really get a Spotify account.” Spotify and other streaming services like Pandora are frequently under attack from artists and their advocates over what they contend are unfairly low royalties or failure to pay. This week Spotify removed thousands of songs from the punk label Victory Records after accusations that it had not paid songwriting royalties. But Mr. Lamb, 39, is an example of a growing class of musicians who are far from superstars — he still has a day job — yet can reap sometimes substantial wages from streaming. The growth of playlists and social media means that an unfamiliar song can pop into a listener’s feed and be heard, saved and shared. Each listen generates a fraction of a penny. The financial viability of streaming is still under constant debate in music circles, especially as streaming begins to replace more lucrative CD and download sales. “Thinking that $40,000 is sufficient compensation for 10 million streams is just absolutely tragic,” said Mike Doughty, a singer-songwriter and the former leader of the band Soul Coughing, who has been a frequent commentator on the problems of the music industry in the digital age. Mr. Lamb benefits from a business infrastructure that lets independent musicians operate outside the standard label system. His music is released through CD Baby, a distributor that charges its customers $49 to carry an album, as well as a 9 percent cut of digital income from stores like iTunes, Spotify and Rhapsody. That arrangement gives musicians a much higher percentage than they would earn through a typical record label contract. Tracy Maddux, the chief executive of CD Baby, said that last year the company paid its artists $55 million for digital uses of their music, and that Mr. Lamb’s story was not unusual. “We have hundreds of clients that make that kind of money in a year,” Mr. Maddux said. Different music streaming services have varying costs, platforms and catalog sizes. See how some of the most popular companies stack up against one another. “There is a whole ecosystem of independent artists that are rethinking the way the business is done,” Mr. Lamb added. “I have friends who make a ton of money off YouTube, and vinyl sales and house shows. There are so many ways to make it work beyond the traditional model.” “Everyone’s Got Something” was released in 2011 as part of Mr. Lamb’s album “Back to You.” Doug Ford, a programmer at Spotify, said the company’s algorithm recommended the song when he was building “Your Favorite Coffeehouse,” a collection meant to evoke “sitting in a comfy chair sipping a latte.” It has become one of Spotify’s most popular playlists, with more than 1.3 million followers. After “Everyone’s Got Something” took off, another of Mr. Lamb’s songs, the more upbeat “Little Bit,” made it onto Spotify’s “Mood Booster” playlists. Together, the two songs have gotten more than 24 million plays. Mr. Lamb said that the royalties from “Everyone’s Got Something” and “Little Bit” have given him “a little safety.” But streaming remains only one part of his career, he said. His day job is working at Sorted Noise, a Nashville firm that specializes in placing songs in television and the movies, a process that Mr. Lamb said has thoroughly permeated his songwriting. “I’m the montage where the guy is walking away in the rain and the girl is crying,” he said. Mr. Lamb grew up in Mississippi and moved to Nashville in 2001, and remained on the fringes of the industry there. But that may have worked to his benefit. Since Mr. Lamb was never signed to a record label or music publisher, he retained full rights to his music. “No one ever offered me anything,” he said. http://www.nytimes.com/2015/10/23/bu...streaming.html ![]() The Scientists Encouraging Online Piracy with a Secret Codeword BBC What if you're a scientist looking for the latest published research on a particular subject, but you can't afford to pay for it? In many countries, it's against the law to download copyrighted material without paying for it - whether it's a music track, a movie, or an academic paper. Published research is protected by the same laws, and access is generally restricted to scientists - or institutions - who subscribe to journals. But some scientists argue that their need to access the latest knowledge justifies flouting the law, and they're using a Twitter hashtag to help pirate scientific papers. Andrea Kuszewski, a cognitive scientist and science writer, invented the tag, which uses a code phrase: "I can haz PDF" - a play on words combining a popular geeky phrase used widely online in a meme involving cat pictures, and a common online file format. "Basically you tweet out a link to the paper that you need, with the hashtag and then your email address," she told BBC Trending radio. "And someone will respond to your email and send it to you." Who might that "someone" be? Kuszewski says scientists who have access to journals, through subscriptions or the institutions they work at, look out for the tag so they can help out colleagues in need. Once contact is made, all subsequent conversation is kept off of social media - instead, scientists correspond via email. The original tweet is deleted, so there's no public record of the paper changing hands. Kuszewski and others say the method is necessary to get up-to-date research in the hands of academics from developing countries, and her and other scientists say they consider the pirating "civil disobedience" against a system that includes for-profit publishing companies. But of course publishers are opposed to free swapping of the papers they publish, and they are usually backed up by the law. Pirating journal articles violates most publishers' terms of service, and is illegal in many jurisdictions. They also argue that it is morally wrong - because by managing the publication and dissemination of scientific research, they are performing a vital function that needs to be paid for. These arguments don't deter Kuszewski, who thinks her hashtag will lead to a change in the way papers are published and accessed. "If we keep finding workarounds to get research to people for free and enough people are doing it, and it causes enough of a ruckus, eventually something will happen to change it," she says. The pirating of academic papers goes beyond the hashtag, however, and sites have been set up where papers can be downloaded for free, often illegally. Elsevier, the Dutch company which publishes The Lancet and many other medical and scientific journals, is suing one such pirate site, Sci-Hub, under US law. Sci-Hub was founded by a Kazakh humanities researcher, Alexandra Elbakya, and has tens of thousands of daily users, many from places like Russia and India. She says she's not concerned about the US case, and denied that swapping academic papers is tantamount to stealing. "I don't think it can be equated very easily to theft. Theft is when you take something and the owner loses possession. But in copyright infringement, you don't take anything from other people," Elbakya says. "Many university researchers need access to these papers because subscriptions are very expensive." Elsevier wouldn't comment on the case, but did give a statement to BBC Trending saying that they recognise that access and publishing options are key for researchers. The company says it provides open access journals, rental options, individual article purchases and other means of disseminating research papers. And just as business models for music have changed in a world of illegal downloading - with streaming sites lowering the cost of legal access - now several publishers are shifting to more open models of accessing research, although Kuszewski believes the changes aren't happening fast enough. "Science moves slow enough as it is," she argues, "so anything that we can do to make it happen faster is a good thing." Reporting by Mukul Devichand and Estelle Doyle http://www.bbc.co.uk/news/blogs-trending-34572462 ![]() Getting a Full PDF from a DRM-Encumbered Online Textbook Jonathon Vogel I recently started a calculus course that uses an online textbook. Buying this textbook online was mandatory, not for the content, but to get an electronic access code for homework assignments. While I had the option of additionally buying a physical copy of the book, I don't like the idea of textbook publishers trying to squeeze the used books market with scummy tactics like this. On top of this, unless I paid extra, I will lose access to this book at some point in the future. That is unacceptable to me. So... I'm going to crack it. (Yes, I probably could have just torrented a PDF copy. But that's no fun!) The DRM on this textbook is pretty intense. Of course, there isn't a "download PDF" option. There is a printing option, but it's limited to 10 pages at a time, and prints the pages out with a large watermark in the center, along with licensing info (my name, number, and a "do not scan, copy, duplicate, distribute, or exercise any freedom with the material" notice) in the margins. Fun! First, we need to download all the pages. Due to the download limit, this is going to take forever... right? Nope! A little Clojure and java.awt.Robot has our mouse pointer whizzing around the screen by itself. Code:
(ns scraper.core) (import '(java.awt Robot) '(java.awt.event KeyEvent InputEvent)) (use '[clojure.string :only (join)]) (defn char-to-key [c] (KeyEvent/getExtendedKeyCodeForChar (int c))) (defn type-key [r c] (.keyPress r c) (.delay r 100) (.keyRelease r c)) (defn type-string [r s] (doseq [c (seq s)] (let [upcase? (Character/isUpperCase c)] (if upcase? (.keyPress r KeyEvent/VK_SHIFT)) (.delay r 100) (type-key r (char-to-key c)) (if upcase? (.keyRelease r KeyEvent/VK_SHIFT)) (.delay r 100))) (.waitForIdle r)) (defn mouse-down [r] (.mousePress r InputEvent/BUTTON1_DOWN_MASK)) (defn mouse-up [r] (.mouseRelease r InputEvent/BUTTON1_DOWN_MASK)) (defn click-mouse [r f] (when-not (nil? f) (f r)) (doto r (mouse-down) (.delay 100) (mouse-up) (.waitForIdle))) (defn to-next-page [r] (.mouseMove r 727 752)) (defn to-print-button [r] (.mouseMove r 758 154)) (defn to-page-range [r] (.mouseMove r 630 406)) (defn to-page-range-box [r] (.mouseMove r 713 401)) (defn type-range [r l u] (type-string r (format "%s-%s" (str l) (str u)))) (defn to-modal-print [r] (.mouseMove r 624 527)) (defn to-save-button [r] (.mouseMove r 285 166)) (defn rename-file [r new-name] (doto r (.delay 1500) (type-key KeyEvent/VK_BACK_SPACE) (type-key KeyEvent/VK_BACK_SPACE) ; just to be sure :^) (.delay 1500) (type-string new-name) (.delay 1500))) (defn to-modal-save [r] (.mouseMove r 1008 734)) (defn open-print-menu [r] (doto r (click-mouse to-print-button) (.delay 3000) (click-mouse to-modal-print) (.delay 1500))) (defn save-file-as [r new-name extra-wait] (Thread/sleep extra-wait) (doto r (click-mouse to-save-button) (.delay 1500) (rename-file new-name) (.delay 1500) (click-mouse to-modal-save))) (defn print-pages-single [r start end prefix] (doseq [n (range start end)] (doto r (open-print-menu) (.delay 1500) (save-file-as (format "%s%04d" prefix n) 1000) (.delay 1500) (click-mouse to-next-page) (.delay 4500)))) (defn print-pages-range [r start end prefix] (doseq [[range-start range-end] (map (juxt first last) (partition 10 10 [] (range start end)))] (let [range-name (format "%s%d-%s%d" prefix range-start prefix range-end) range-file-name (format "%s%04d_%s%04d" prefix range-start prefix range-end)] (doto r (click-mouse to-print-button) (.delay 3000) (click-mouse to-page-range) (.delay 1500) (click-mouse to-page-range-box) (.delay 1500) (type-string range-name) (.delay 1500) (click-mouse to-modal-print) (.delay 3000) (save-file-as range-file-name 10000) (.delay 4500))))) (defn print-pages [r page-ranges] (doseq [s page-ranges] (let [[start end prefix single?] s] ((if single? print-pages-single print-pages-range) start end prefix)))) (let [r (Robot.)] (println "Starting in 1 second...") (doto r (print-pages-single 0 1 "0000_cover") (print-pages-single 1 30 "0000_prologue") (print-pages-range 1 1171 "") (print-pages-range 1 147 "A") (print-pages-range 1 11 "R") (.waitForIdle))) Now, we need to get a page into image form, so we can play around with it in GIMP. Once we get a process worked out, we can automate it with ImageMagick and process all 1500-odd pages of the book. Getting this image is easy: pick a page and run: Code:
convert -density 300 -quality 100 $PDF -crop 1846x2306+208+322 out.png Luckily page R11 was totally white, so converting it to a PNG yielded a clean, isolated copy of the watermark. Dealing with the margins will be easy, we can just crop them out, so lets focus on the watermark. I made everything but the watermark itself transparent in GIMP, and after that removing it from the original image is as simple as overlaying the cleaned-up version on the page and setting the watermark's layer mode to divide. Now we need to repeat our earlier PDF->PNG conversion for all the files. This wasn't much harder with a dash of GNU parallel (an incredibly handy tool): Code:
parallel convert -density 300 -quality 100 {} '~/book-imgs/{/.}-%d.png' ::: pdfs/*.pdf Code:
convert test.png watermark.png -compose Divide_Src -composite out.png And we have a nice, clean page... which I'd love to show you, but, copyrights. So just image a pristine (well, there's a few artifacts) textbook page, with no ugly watermarks. Ahhhh. Now, lets do this 1500 times! Time for reach for parallel again: Code:
parallel convert {} -background white -fill white -draw \"rectangle 160,163 1799,215\" -draw \"rectangle 160,2725 2342,2834\" -flatten watermark.png -compose Divide_Src -composite \"/home/jon/book-imgs-proc/{/}\" ::: ~/book-imgs/*.png This command seems a bit confusing, so I'll break it into its constituent pieces. The first part is -background white ... -flatten, which fills the transparent edges with white. I wanted the images to have an 8.5x11 ratio (because I'm a silly American), and it turns out they already were in that ratio - if I included the transparent part. No cropping required! The next part is -fill white -draw \"rectangle 160,163 1799,215\" -draw \"rectangle 160,2725 2342,2834\". Since we aren't cropping out the licensing text, I'm instead simply covering it up with some filled-white rectangles. The coordinates took a bit of tweaking, but it worked out pretty well. Finally after we -flatten, and all the operations have happened on the source image, we can load the watermark image and divide by it as before: watermark.png -compose Divide_Src -composite \"/home/jon/book-imgs-proc/{/}\". To make everything work, I had to manually move the watermark up in GIMP to get it to align. Not sure why, but after I did this everything processed basically perfectly (it's still not exactly aligned, so there's some very thin gray lines, but it's good enough for me). Now, we can use convert again (ImageMagick is so useful) to join the PNG images into a single PDF: Code:
convert ~/book-images-proc/*.png book.pdf Code:
nice -n 19 convert -limit area 1GiB -limit memory 1GiB -limit map 1GiB ~/book-imgs-proc/*.png book.pdf Now we can use basically any PDF OCR tool to make the text searchable. If I had the motivation I could probably scrape the original text from the book to get it perfect, but I don't care that much, so OCR it is. I already have Ruby and the Tesseract OCR engine installed, so I just grabbed the one-script pdfocr tool from its Github repo. One extra command installation it needed for some reason and... Code:
./pdfocr -t -i book.pdf -o book-ocr.pdf http://vgel.me/posts/cracking-online-textbook/ ![]() Rovi and Michigan State University Establish Largest U.S. Library Media Collection Rovi donates rare archive of CDs, Blu-Rays, DVDs and video games spanning two decades of entertainment Press release Rovi Corporation and Michigan State University today announced that Rovi has donated a rare and valuable media collection to MSU. This donation establishes the largest media collection held by a library in the United States. The new “Rovi Media Collection” is comprised of close to one million CDs, Blu-Rays, DVDs, and video games, and is now publicly available through the MSU library and interlibrary loan services. “We are honored to be the proprietors of the largest media archive in the country, which has quickly become the most requested material in the Michigan inter-library loan system,” said Clifford H. Haka, director of libraries, Michigan State University. “The ‘Rovi Media Collection’ dramatically enhances our teaching curriculum and research within the College of Music, popular culture and film studies, and an emerging gaming program. Assembling a collection of such cultural and historic importance and overall magnitude would simply not have been feasible with our current budget. On behalf of all of our users at MSU and across Michigan, we thank Rovi for this generous gift.” “Rovi’s extensive collection of entertainment media has enabled the cataloging, metadata tagging, and editorial description of numerous albums, artists, movies, TV shows, and video games for our customers worldwide,” said Kathy Weidman, senior vice president and general manager, metadata, Rovi. “Our donation to Michigan State University marks the passing of these materials for use in an educational capacity, while also providing accessibility to the general public. It is extremely gratifying to establish the largest collection of entertainment content in a U.S. library and allow future generations to access and enjoy these wonderful media artifacts from the past two decades.” As a result of Rovi’s meticulous efforts to catalog and ingest metadata about entertainment media products for its customers, the company’s archive has grown exponentially over the past 20 plus years. As one of the nation’s top 50 libraries with more than five million volumes held, MSU will manage the on-going care, storage, and maintenance of the physical collection. The archive consists of CDs that have been commercially available in the U.S. as well as a significant percentage of releases imported from Europe. The DVD collection, which was started the year DVDs were introduced to the market, similarly represents the vast majority of commercially released DVDs in the U.S. The games archive focuses primarily on console games and PC games, beginning in 1999, and includes a number of titles that date back to the early 80s. The "Rovi Media Collection" also includes a catalog of metadata elements consisting of nearly 20 million data points. Today, Rovi continues to expand its metadata business and is a leading resource for international metadata with availability in more than 70 countries. http://www.marketwatch.com/story/rov...ion-2015-10-19 ![]() Western Digital To Aquire SanDisk For $19 Billion Matt Burns Western Digital just bought a bunch of memory cards. The storage giant just announced that it has agreed to buy SanDisk Corp for about $19 billion. This comes after speculation that SanDisk was shopping for a buyer. The deal values SanDisk at $86.50 a share, which is a 15% premium on the previous day’s closing price, giving the company a value of $15.4 billion. SanDisk is currently up 4.78% in pre-market trading. This deal brings together two of the largest storage companies. In many ways, SanDisk, with its deep investment in flash memory chips represents a future without the spinning disk hard drives of Western Digital. Yet despite early explosive growth, SanDisk as of late reported results that have lagged behind expectations. Likewise, Western Digital has watched its core business slip away as the industry moves towards flash memory. SanDisk CEO Sanjay Mehrotra is expected to join the Western Digital board upon the closing of the deal. Western Digital CEO Steve Milligan will continue in that role. http://techcrunch.com/2015/10/21/wes...for-19-billion ![]() Some Popular 'Self Encrypting' Hard Drives Have Really Bad Encryption Joseph Cox So you’ve bought your encrypted hard drive to protect your sensitive files from prying eyes. But is the crypto on your device really as secure as you think? Researchers claim one of the more popular brands of so-called “self-encrypting” drives is plagued by serious security vulnerabilities that allow an attacker trivial access to data stored on its products. The problems relate to Western Digital's line of “My Passport” hard drives. Some of them seem pretty popular; one model has nearly 2,000 ratings on Amazon. The details were included in a paper dated 28 September, and posted to the Full Disclosure email list this week—a space where vulnerability researchers post their findings if the affected company is not being cooperative. As well as some other vulnerabilities, the researchers write that they discovered “backdoors on some of these devices, resulting in decrypted user data, without the knowledge of any user credentials.” The My Passport drives allow a user to set a password in order to use them, so theoretically anyone who stole the device wouldn't be able to get to the files on it without the right code. “It turns out that's not really true,” Matthew Green, assistant professor at Johns Hopkins University told Motherboard in an email. “The authors show that due to a tragicomedy of errors on the part of [Western Digital], the security of the drives is actually very weak.” According to Green, the worst of the problems is how the encryption keys are generated. “[Western Digital] does it using the C rand() function, which is known not to be cryptographically secure,” he wrote. Rand() is a very simple command for returning a pseudo-random number, and is not up to the task of producing a suitably strong key for keeping data secure. On top of this, the key is seeded with the time it was created in a 32-bit format. “That means instead of requiring billions of years to crack, an attacker who steals your drive can guess the key in a short time using a single PC,” Green added. After all that, it turns out that some models just store the password on the hard drive anyway. That means an attacker wouldn’t even need your password to break into the device. “That doesn't even make sense. That key should never be stored on the drive,” Green added.“This is a big problem for people who are relying on them, or worse, are using them to meet regulatory requirements such as encrypting health information for HIPAA.” The Health Insurance Portability and Accountability Act is a piece of US federal legislation that in part is designed to protect health care data. On the Full Disclosure post, the research authors claim that Western Digital has been informed of the vulnerabilities, and say they are not aware of any patches to these problems. A Western Digital spokesperson said that the company “has been in a dialogue with independent security researchers relating to their security observations in certain models of our My Passport hard drives.” The spokesperson added that “We continue to evaluate the observations,” but would not answer directly whether the company intended to issue a patch. They also did not say how such a patch would reach all of its affected customers. “We highly value and encourage this kind of responsible community engagement because it ultimately benefits our customers by making our products better. We encourage all security researchers to responsibly report potential security vulnerabilities or concerns to WD Customer Service and Support.” Regardless, “There is no way to look at this security design and say that it was well thought out by expert security engineers,” Green said. “Until the flaws are verifiably fixed, these devices should be viewed as effectively unencrypted. Hopefully nobody is using them for anything that really matters.” http://motherboard.vice.com/read/som...bad-encryption ![]() Einstein Wouldn't Like it: New Test Proves Universe is "Spooky" Ben Hirschler The universe really is weird, which is bad news both for Albert Einstein and for would-be hackers hoping to break into quantum encryption systems. Eighty years after the physicist dismissed as "spooky" the idea that simply observing one particle could instantly change another far-away object, Dutch scientists said on Wednesday they had proved decisively that the effect was real. Writing in the journal Nature, researchers detailed an experiment showing how two electrons at separate locations 1.3 km (0.8 mile) apart on the Delft University of Technology campus demonstrated a clear, invisible and instantaneous connection. Importantly, the new study closed loopholes in earlier tests that had left some doubt as to whether the eerie connection predicted by quantum theory was real or not. Einstein famously insisted in a 1935 scientific paper that what he called "spooky action at a distance" had to be wrong and there must be undiscovered properties of particles to explain such counter-intuitive behavior. The idea certainly confounds our day-to-day experience of the world, where change only appears to occur through local interactions. But in recent decades scientific evidence has been building that particles can indeed become "entangled", so that no matter how far apart they are, they will always be connected. The Delft experiment is conclusive because, for the first time, scientists have closed two potential loopholes at once. The first suggests that particles could somehow synchronize behavior ahead of time, while the second implies that testing might detect only a subset of prepared entangled pairs. To prove their case, the team led by Delft professor Ronald Hanson used two diamonds containing tiny traps for electrons with a magnetic property called spin and measured all entangled pairs across 1.3 km separating two laboratories. The experiment effectively closes a chapter in an 80-year scientific debate, but Hanson said it also had important implications for the future, since sophisticated cryptography is already using quantum properties to guarantee data security. Such quantum encryption systems will only be 100 percent secure, however, if all loopholes are closed, as in the Delft system. "Loopholes can be backdoors into systems," Hanson told Reuters. "When you go loophole-free then you add an extra layer of security and you can be absolutely certain there is no way for hackers to get in." (Editing by Hugh Lawson) http://uk.reuters.com/article/2015/1...0SF2GQ20151021 ![]() New Laws to Allow Spies to Hack into Smartphones and Computers ‘to be Introduced in the Coming Weeks’ The Government has pledged to bring back major powers to Britain’s spying agencies Andrew Griffin Britain’s spies are about to be given huge new powers that will allow them to look in on people’s phones and computers, according to reports. A revived and re-named version of the hugely-controversial “Snoopers’ Charter” is set to give spies a “dizzying” range of surveillance and hacking powers, The Times has reported. The new legislation will be introduced next month, the paper reported. The new powers will please MI5, MI6 and GCHQ, which have said in the past that they lack the powers to be able to protect the country against threats. But they are likely to anger privacy campaigners, many of whom united to defeat the Snoopers’ Charter when it was first presented. The new powers could include giving Britain’s spying agencies the power to take over a phone remotely and access all of the documents – including text messages and emails – and photos that are stored on it. They will then be able to install software that will allow them to look in on the messages and data of people at any time, according to reports. Earlier this year, a major report recommended that the UK should completely overhaul the law that regulates the powers that spies have to intercept people’s communications. The new legislation will partly respond to those problems with the current regulation – but will also introduce huge new powers allowing people to spy on targets with little restriction, according to the reports. First look inside GCHQ The new powers will also partly work to bring back some of the powers of the Snoopers’ Charter. That law was defeated by the Liberal Democrats during the last government, but the Conservatives indicated almost as soon as they were elected that they would look to revive it. http://www.independent.co.uk/life-st...-a6702301.html ![]() Android 6.0 Re-Implements Mandatory Storage Encryption for New Devices As long as you meet the minimum speed requirements, that is. Andrew Cunningham Shortly after the announcement of iOS 8 in 2014, Google made headlines by saying that it would make full-device encryption mandatory for new Android devices running version 5.0. It then made more headlines several months later when we discovered that the company backed down, "strongly recommending" that Android device makers enable encryption but stopping short of actually requiring it. Now Google has published an updated version of the Android Compatibility Definition Document (PDF) for Android 6.0, and it looks like mandatory encryption is back with a couple of exceptions. New devices that come with Marshmallow and have AES crypto performance above 50MiB-per-second need to support encryption of the private user data partition (/data) and the public data partition (/sdcard). The relevant portion of the document, emphasis ours: 9.9. Full-Disk Encryption If the device implementation supports a secure lock screen reporting "true" for KeyguardManager.isDeviceSecure(), and is not a device with restricted memory as reported through the ActivityManager.isLowRamDevice() method, then the device MUST support full-disk encryption of the application private data (/data partition), as well as the application shared storage partition (/sdcard partition) if it is a permanent, non-removable part of the device. For device implementations supporting full-disk encryption and with Advanced Encryption Standard (AES) crypto performance above 50MiB/sec, the full-disk encryption MUST be enabled by default at the time the user has completed the out-of-box setup experience. If a device implementation is already launched on an earlier Android version with full-disk encryption disabled by default, such a device cannot meet the requirement through a system software update and thus MAY be exempted. The new rule continues to exempt phones and tablets that were launched with older versions of Android and upgraded to Marshmallow later. It also doesn't apply to devices that don't meet the minimum crypto performance requirements. This will mostly cover cheap devices with low-end 32-bit SoCs that lack dedicated hardware acceleration for encryption and decryption (or the crypto speed boosts that come from using 64-bit hardware with the ARMv8 instruction set). And devices without lock screens (think Android Wear) that ship with Android 6.0 won't require encryption by default, either. All in all this is a step forward, though it includes a few too many loopholes for our liking and does nothing to protect users of most of the Android phones being sold and used today. There are many more changes in the CDD, and our own Ron Amadeo is comparing the new guide to the one for Android 5.1 in order to publish a more detailed report soon. http://arstechnica.com/gadgets/2015/...r-new-devices/ ![]() Apple Tells U.S. Judge 'Impossible' to Unlock New iPhones Nate Raymond Apple Inc (AAPL.O) told a U.S. judge that accessing data stored on a locked iPhone would be "impossible" with devices using its latest operating system, but the company has the "technical ability" to help law enforcement unlock older phones. Apple's position was laid out in a brief filed late Monday, after a federal magistrate judge in Brooklyn, New York, sought its input as he weighed a U.S. Justice Department request to force the company to help authorities access a seized iPhone during an investigation. In court papers, Apple said that for the 90 percent of its devices running iOS 8 or higher, granting the Justice Department's request "would be impossible to perform" after it strengthened encryption methods. Those devices include a feature that prevents anyone without the device's passcode from accessing its data, including Apple itself. The feature was adopted in 2014 amid heightened privacy concerns following leaks by former National Security Agency contractor Edward Snowden about NSA surveillance programs. Apple told U.S. Magistrate Judge James Orenstein it could access the 10 percent of its devices that continue to use older systems, including the one at issue in the case. But it urged the judge to not require it to comply with the Justice Department's request. "Forcing Apple to extract data in this case, absent clear legal authority to do so, could threaten the trust between Apple and its customers and substantially tarnish the Apple brand," Apple's lawyers wrote. A spokeswoman for Brooklyn U.S. Attorney Robert Capers, whose office is handling the case, declined comment. Earlier this month, Orenstein expressed skepticism about whether he could require Apple to disable security on the iPhone, citing Congress' failure to act on the issue of encryption despite the urging of the Justice Department and Federal Bureau of Investigation. Orenstein deferred ruling until Apple had a chance to say if it was "technically feasible and, if so, whether compliance with the proposed order would be unduly burdensome." Apple in its brief said it limited its views to those questions rather than the broader legal issue at hand, which it called "important." In an order Tuesday, Orenstein invited Apple to address that issue. A hearing is scheduled for Thursday. The case is In re Order requiring Apple, Inc to assist in the execution of a search warrant issued by the court, U.S. District Court, Eastern District of New York, No. 15-mc-01902. (Reporting by Nate Raymond in New York; Editing by Marguerita Choy and Michael Perry) http://www.reuters.com/article/2015/...0SE2NF20151021 ![]() DOJ Dismisses Apple's Arguments Against Decrypting iOS Communications Dell Cameron The U.S. government rejected Apple’s arguments in federal court that unlocking iOS devices for police would damage the tech giant’s public image and overburden its employees and resources. Federal courts should require Apple to unlock encrypted data because the operating system is “licensed, not sold,” to customers, the Justice Department argued in a reply brief in the U.S. District Court for the Eastern District of New York. “Apple designed, manufactured, and sold [the phone] that is the subject of the search warrant,” the government told U.S. Magistrate Judge James Orenstein. “But that is only the beginning of Apple’s relationship to the phone and to this matter. Apple wrote and owns the software that runs the phone, and this software is thwarting the execution of the warrant.” The Justice Department filing, in a case involving an iPhone discovered on a suspect indicted for methamphetamine possession, reflects the broader battle between government and industry over law-enforcement access to encrypted communications. This debate, known as the "Crypto Wars," has pitted technology companies and privacy groups against police and intelligence agencies since at least the 1990s. The government noted in its reply that Apple has openly admitted that it licenses iOS, meaning it does not fully transfer the attendant rights and responsibilities of ownership to its customers. “Apple’s software licensing agreement specifies that iOS 7 software is ‘licensed, not sold’ and that users are merely granted “a limited non-exclusive license to use the iOS Software,” the brief said. Apple filed a brief of its own on Monday saying that, “in most cases now and in the future,” it would be “impossible” for it to extract encrypted data from an iOS device. Every device running at least iOS 8, released in late 2014, encrypts and stores all of its communications in a way that Apple cannot reverse-engineer. But the defendant in the New York drug case is not among the approximately 90 percent of Apple device owners running at least iOS 8, so Apple retains the technical ability to provide the suspect’s communications to police. But Apple argued in its brief that, while decrypting a single phone was not particularly onerous, doing so would set a precedent that would unduly burden the company in the future. Routinely aiding law enforcement, the company said, would substantially tax its resources, diverting employees, software, and equipment from daily operations. “This burden,” it argued, “increases as the number of government requests increases.” The Justice Department dismissed this argument. “Apple asserts that its burden ‘increases as the number of government requests increases,’” the government replied on Thursday, “but it makes no attempt to quantify this burden or demonstrate that such orders have in fact cumulatively burdened it significantly. “To the contrary, Apple demonstrates why any cumulative burden is minimal and likely to decrease with regard to the type of relief requested here: by its own measure, Apple retains the ability to bypass the passcode on only the 10 percent of its mobile devices that are ‘pre-iOS 8,’ and that number will continue to shrink as new devices are upgraded and replaced.” Robert Capers, the author of the Justice Department's brief, has been nominated by President Barack Obama to succeed Loretta Lynch, now Obama's attorney general, as the U.S. Attorney for the Eastern District, which includes Brooklyn, Queens, and Staten Island. Apple had also argued that decrypting user data for police would damage its reputation at a time when “public sensitivity to issues regarding digital privacy and security is at an unprecedented level.” The harm to Apple’s reputation and its relationship with its customers, it said, “could have a longer term economic impact beyond the mere cost of performing the single extraction at issue.” The government rejected this argument, saying that Apple offered no concrete evidence that reputational concerns constituted an “undue burden” as defined by law. Andrew Crocker, a staff attorney at the Electronic Frontier Foundation, which has filed a brief in the case, said it's irrational for the court to compel Apple to turn over information it doesn't already possess. “This is just the government sort of grasping at reasons why Apple is somehow already involved in the case,” he told the Daily Dot by phone. The Justice Department's flat-out rejection of Apple's brand-integrity concerns was dismissed too hastily, says Crocker. “The government sort of waved its hand and said ‘those aren't relevant concerns here.’” “The government's argument is essentially that Apple has never really objected in this case, and in fact they've set up guidelines by which they'll comply—and that might be true—but that doesn't stop Apple from objecting in any particular case,” Crocker added. “They have the ability to do that here, and clearly the judge was concerned about it too.” In addition, Apple warned that, if it were forced to routinely decrypt data for law enforcement, its employees could theoretically be subpoenaed in every case to provide expert testimony and endure cross-examination in court. The company cited a 2012 federal appeals-court case, U.S. v. Cameron, in which a three-judge panel ruled that a convicted child pornographer’s Sixth Amendment rights were violated when he was prevented from confronting in court the Yahoo employee who originally discovered his illicit material. The judges subsequently vacated the district court’s judgement, reversing six of the defendant’s counts. Apple did not immediately respond to a request for comment. http://www.dailydot.com/politics/app...ing-doj-reply/ ![]() Apple Bans Hundreds of App Store Apps after they were Found To be Spying on Users The special software allowed people to look in on phones — and developers might not even have known it was there Andrew Griffin Apple's App Store recent ran a special promotion to coincide with World Aids Day Getty Images Over 250 App Store apps that could look at what people were doing with their phones have been removed by Apple, after they were discovered to have been using software from an advertising company that secretly stole users’ personal information. The apps used a special piece of software that allowed them to harvest users’ email addresses, serial numbers and to get a full collection of what apps people had installed on their phone. Apple took action soon after the problem was discovered. Apple has a range of app review processes, which are supposed to vet new software before it gets into the store to ensure that it is not malicious. But the apps appear to have found a way around that. The software was made by Chinese advertising firm Youmi. It used special techniques to get around Apple’s rules and review process, according to SourceDNA, which found the bug. App makers probably didn’t know that the Youmi software was stealing their information, according to SourceDNA. Apple said that it would work with developers to help them have the software removed and get their apps back up on the store. “We’ve identified a group of apps that are using a third-party advertising SDK, developed by Youmi, a mobile advertising provider, that uses private APIs to gather private information, such as user email addresses and device identifiers, and route data to its company server,” Apple told The Verge. iPhone app store attacked “This is a violation of our security and privacy guidelines. The apps using Youmi’s SDK have been removed from the App Store and any new apps submitted to the App Store using this SDK will be rejected. We are working closely with developers to help them get updated versions of their apps that are safe for customers and in compliance with our guidelines back in the App Store quickly.” http://www.independent.co.uk/life-st...-a6700911.html ![]() Apple and Dropbox Say they Don’t Support a Key Cybersecurity Bill, Days Before a Crucial Vote Brian Fung Apple and Dropbox said Tuesday that they do not support a controversial cybersecurity bill that, according to critics, would give the government sweeping new powers to spy on Americans in the name of protecting them from hackers. The announcement by the two companies comes days before the Senate expects to vote on the legislation, known as the Cybersecurity Information Sharing Act, or CISA. "We don't support the current CISA proposal," Apple said in a statement. "The trust of our customers means everything to us and we don't believe security should come at the expense of their privacy." Dropbox said that the bill needed more privacy protections in order to win its support. "While it’s important for the public and private sector to share relevant data about emerging threats," said Amber Cottle, head of Dropbox global public policy and government affairs, "that type of collaboration should not come at the expense of users’ privacy." Apple and Dropbox join a number of tech companies who say they're against the bill. In recent days, Yelp, reddit, Twitter and the Wikimedia Foundation — which runs Wikipedia — have all said that they oppose CISA. Congress is trying to pass a "cyber security" bill that threatens your privacy. Join us & others to oppose: http://t.co/WtpEoS4ESS — Yelp (@Yelp) October 19, 2015 Other Silicon Valley firms including Google, Facebook and Yahoo have voiced their concerns about the bill through a trade group that represents them in Washington called the Computer and Communications Industry Association. The two firms' entry into the debate — particularly Apple, which rarely wades into Washington policy fights — complicates last-minute efforts to pass the bill, which has bipartisan backing and is expected to get a vote next Tuesday. One of its co-sponsors, Dianne Feinstein, (D-Calif.), the ranking member on the Senate Intelligence Committee, said Tuesday that the bill simply allows companies to share information on “cyber threats” with the government — not personal data. "A bank would not be able to share a customer's name or account information," Feinstein said. "Things like Social Security numbers, addresses, passwords and credit information would be unrelated to a cyber threat and would, except in very exceptional circumstances, be removed" before being transmitted to authorities. But a major critic of the legislation, Sen. Ron Wyden (D-Ore.), said the sheer number of tech companies aligned against the bill shows that it still lacks sufficient privacy safeguards. "Sharing information about cybersecurity threats is a worthy goal," said Wyden. "Yet if you share more information without strong privacy protections, millions of Americans will say, 'That is not a cybersecurity bill. It is a surveillance bill.' " Still, CISA's supporters estimate they have roughly 70 votes in the Senate, enough to approve the White House-backed legislation. Apple has positioned itself aggressively on user privacy, encrypting messages between iPhone users and critiquing the government for asking the company to stop doing so. https://www.washingtonpost.com/news/...-crucial-vote/ ![]() Cybersecurity: Senate Takes Initial Step to Bill's Passage Tami Abdollah The Senate is set to pass a bill aimed at improving cybersecurity by encouraging the sharing of threat information among companies and the U.S. government. An 83-14 procedural vote Thursday represented a healthy endorsement of a bill opposed by companies such as Apple and Dropbox, who said it lacks key privacy protections and may result in personal information ending up in the government's hands. The Cybersecurity Information Sharing Act is co-sponsored by Sen. Dianne Feinstein, D-Calif., and Sen. Richard Burr, R-N.C., who said it was critical to limit increasingly high-profile cyberattacks, such as one suffered by Sony Pictures last year. "This is a good bill. It is a first step. It's not going to prevent all cyberattacks or penetrations, but it will allow companies to share information about the cyber threats they see and the defensive measures to implement to protect their networks," Feinstein said. She said the same tactics are used repeatedly against different targets, which shouldn't happen. More than 21 million Americans recently had their personal information stolen when the Office of Personnel Management was hacked in what that the U.S. believes was a Chinese espionage operation. Companies would receive legal protections from antitrust and consumer privacy liabilities for participating in the voluntary program. Sen. Ron Wyden, D-Ore., who opposed the bill, said it provides liability protection for companies, which are required to remove personal and unrelated information provided to the government only if they know it is personal and unrelated. "How would they know? ...They're required to virtually do no looking. It is the most cursory review," Wyden said. He said Americans would call it "another surveillance bill." The U.S. and the technology industry already operate groups intended to improve sharing of information among the government and businesses, including the Homeland Security Department's U.S. Computer Emergency Readiness Team. The White House said in a statement that it supports the bill, which is part of larger suite of legislation that's needed to provide necessary tools to fight cyber threats and create consistent notification standards for breaches of personal data. "In addition to updating information sharing statutes, the Congress should incorporate privacy, confidentiality protection, and civil liberties safeguards into all aspects of cybersecurity legislation," the White House said. The Senate's vote on final passage is expected next week. The House passed its version of the bill earlier this year with strong bipartisan support. If the Senate were to pass the bill on Tuesday, the two versions would need to be reconciled before being sent to the White House for the president's signature. http://hosted.ap.org/dynamic/stories...10-22-12-46-04 ![]() MEGA is Genius Harlan Lieberman-Berg Let me say this right off of the bat: I'm not weighing in on the business model or the controversies of MEGA when I say that its model is genius. What I am specifically talking about is only its security model. MEGA uses an often misunderstood scheme of encryption as part of its threat model and mitigation that is nothing short of brilliant. This isn't, of course, a new scheme - plenty of others have done similar in the past (Spideroak, for example, among many others.) Still, it's brilliant. Before we delve into the specifics of MEGA, let's work with a hypothetical example that can better show some of the balancing act that is involved, not just between security and usability, but also with whose security we are discussing. Imagine a particular kind of chat service that enables pseudonymous conversations between two, randomly paired users of the service. These users can have a conversation of whatever length they desire, and at the option of either of the users, be rematched to a new random pairing. Let's set out some of the attributes that we would like this service to have. 1. Only the sender and recipient of a message can access and read that message. 2. Users cannot send messages that have no useful purpose - aka, spam. Even with that very limited set of constraints, we run face-first into an immediate conflict. The usability demand ("no spam") requires that the server operator can, in some way, monitor the content of the messages, flying in the face of the demand for privacy ("no spying"). How can we resolve this? Thinking a bit more about what the purposes of these restrictions are, let's break these constraints out a bit more to see if we can come up with an area that we are OK with compromising on. When designing the threat model, there are two different sets of reasons that we might want to ensure that the server operator is unable to read the messages of its users. On the one hand, there is the simple case - we want users to have privacy and for their messages to be secure. Simple. But there is a second, hidden reason that we might want to have this property: we, as the server operator, don't want to be put into a position where we /have/ to monitor our users messages. Let's assume for a second that we simply off-load the burden for securing the messages onto our users, in the same way that email providers do. The users can use PGP or similar for whatever messages they want to remain private, and for messages that they don't care, they can just use plaintext. (Ignoring, for a moment, the problems of actually educating the users on how to use PGP or similar.) Let's say that a user ends up engaging in some kind of criminal activity using this service, somehow - harassment, maybe, or something else undesirable. It is quite reasonable to assume that the investigating officers will come knocking on the server operator's door with a subpoena, wanting whatever records that the server operator has. Or, more dramatic, they may want to wiretap those users messages into the future. Another possibility, a copyright owner may want us to block certain copyrighted content from being shared on the network. This is, for many reasons, not a great position to be in as the server operator! We are having to do a bunch of extra work, both political and technological, including some that might conflict with personal beliefs. Let's make this explicit in our goals: 1. Users can only access messages for which they are the sender or recipient. 2. The server operator cannot monitor the content of any user messages. 3. Users cannot send messages that have no useful purpose - aka, spam. Now, it is clear that the real conflict in goals here is between the latter two, not the former. How can we resolve this conflict? We will need to weaken one of our security goals in order to resolve this problem. How can we do so, without trapping us in the earlier bind? Let's think about the goals of our attacker - in this case, the spammer. • A spammer wants to send their messages to as many users as possible. • A spammer wants either to: o gain some monetary advantage from the messages, even if indirectly ("commercial spam") o disrupt the service, either through technological means or by scaring off legitimate users ("lulz spam") Clearly, regardless of the reason, spammers are incentivized to not have extended conversations with the people they are communicating with. They wield a shotgun, not a sniper rifle. This gives us multiple advantages - not only can we apply technological mechanisms such as rate limiting to interfere with their aims, we have a potential option for weakening one of our security goals without causing a major loss. Since there is not a substantial amount of "metadata" to be leaked in a service that's randomly pairing users - since they can't choose who they are communicating with - our core goals around privacy of the users is aimed at the content of the messages. For spammers, who are likely to want to spew their message and re-pair as fast as possible, it means there is not a whole lot of value to communicating for an extended period of time (thus causing messages to contain more and more private information). How about structuring our goals to recognize that? 1. Users can only access messages for which they are the sender or recipient. 2. The server operator cannot monitor the content of any user messages, unless one of the participants chooses to enable them to. 3. Users cannot send messages that have no useful purpose - aka spam - without detection. If messages can be revealed by one of the participating users - with, say, a "Report" button - the server operator is able to gain information about people misusing the service and take action against them, without having to risk a broad ability to access the content of messages. With this weakened set of goals, there is a pretty clear path towards how one can construct a system to achieve them. Let's turn our attention back to MEGA. MEGA is the successor to Megaupload, which was taken down after a Department of Justice investigation into criminal copyright infringement in 2012. One of the prime elements of the controversy around Megaupload was that they had tools in place to seek out certain kinds of content across the network and remove it, regardless of the links to it (as they did with child pornography), and yet they refused to use those techniques to remove copyrighted files. Their ability to access the content that was uploaded to their system caused them to be put into a position where they were being forced to use it - or suffer the consequences. MEGA, its successor, advertises that it uses client-side encryption to ensure that they cannot read the content of the files which are uploaded to them. This, however, is not to ensure user privacy. In the (vastly) most common usecase, the encryption key is simply provided to those who would access the files as part of the URL and publicly posted. It is obvious that linking to a file, unencrypted, at the path example.com/fileA is no more or less secure than linking to a file, encrypted, at the path example.com/fileA#encryptionkey -- _for the users. It is, however, a tremendous difference to the server operator's ability to make statements about whether or not they are able to access the content stored on their system. This is a "hack" in the traditional sense, twisting the normal purpose of encrypting prior to upload -- to make the information accessible only to authorized people -- into a different security model altogether, one that blinds only a single person -- the operator -- to the content. For better or worse, MEGA is genius. https://blog.setec.io/articles/2015/10/18/mega.html Until next week, - js. ![]() Current Week In Review ![]() Recent WiRs - October 17th, October 10th, October 3rd, September 26th Jack Spratts' Week In Review is published every Friday. Submit letters, articles, press releases, comments, questions etc. in plain text English to jackspratts (at) lycos (dot) com. Submission deadlines are Thursdays @ 1400 UTC. Please include contact info. The right to publish all remarks is reserved. "The First Amendment rests on the assumption that the widest possible dissemination of information from diverse and antagonistic sources is essential to the welfare of the public." - Hugo Black |
![]() |
![]() |
![]() |
Thread Tools | Search this Thread |
Display Modes | |
|
|
![]() |
||||
Thread | Thread Starter | Forum | Replies | Last Post |
Peer-To-Peer News - The Week In Review - July 16th, '11 | JackSpratts | Peer to Peer | 0 | 13-07-11 06:43 AM |
Peer-To-Peer News - The Week In Review - July 9th, '11 | JackSpratts | Peer to Peer | 0 | 06-07-11 05:36 AM |
Peer-To-Peer News - The Week In Review - January 30th, '10 | JackSpratts | Peer to Peer | 0 | 27-01-10 07:49 AM |
Peer-To-Peer News - The Week In Review - January 16th, '10 | JackSpratts | Peer to Peer | 0 | 13-01-10 09:02 AM |
Peer-To-Peer News - The Week In Review - December 5th, '09 | JackSpratts | Peer to Peer | 0 | 02-12-09 08:32 AM |