tag:blogger.com,1999:blog-86556562024-02-20T05:31:36.064-08:00QuintharDavid Barrett's blog.David Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.comBlogger104125tag:blogger.com,1999:blog-8655656.post-7849126683305214542014-01-10T06:15:00.000-08:002014-01-10T06:15:34.261-08:00How to build a system to "Opt Out" of warrantless wiretappingI love the <a href="http://lists.zooko.com/mailman/listinfo/p2p-hackers">p2p-hackers mailing list</a>. So many smart people talking about so many cool things. The conversation has recently turned to how to build a VoIP system that would bypass the NSA's warrantless wiretapping to the greatest possible degree, while still being usable in the real world. Here's my proposal:<br />
<br />
--<br />
<br />
I think the challenge with building a new phone system is that the existing phone systems are already amazing. Skype filled a void for cheap long distance. But now that that void is filled, and you can call anybody in the world from your phone easily (unlimited nationwide plans are common, and Skype covers the rest of the world), it'll be very difficult for any new voice service to take root. But I could imagine it happening if:<br />
<br />
1) It's backwards compatible with the current voice services (eg, you can call anybody with it, and they can call you, regardless of whether you use the service)<br />
<br />
2) It offers tangible value to you even if the person you're calling isn't using it<br />
<br />
3) It offers tangible value to you even if the people calling you don't use it<br />
<br />
4) Those values increase as the number of people who use it increases<br />
<br />
5) It automatically advertises itself<br />
<br />
With these, then you can fully adopt this service -- without any downsides -- and gain value from it regardless of whether anybody else does. Furthermore, the value increases as the network size increases, so you have an incentive to encourage others to use it. As for what that service might be, that's a tall bar. But I could imagine protection against dragnet-style government surveillance being compelling to a certain demographic.<br />
<br />
As for how that might work, that's tough. But imagine a new VoIP client like the old Skype (eg, P2P with a distributed relay service for NATs/firewalls), except truly encrypted. That would be pretty straightforward to do: the audio/video codecs are pretty refined, and there are great P2P libraries ready to go. The problem is: nobody is using is, so you have no reason to use it either.<br />
<br />
But what if everybody registered their "real" phone number with some DHT, and then coupled this app with a collection of VoIP->POTS (Plain Old Telephone System) gateways. So when I type in your phone number, first it checks to see if I can use this secure system, and contacts you directly via VoIP. But if you aren't in the system, it just calls you via a POTS gateway.<br />
<br />
Ok, so now we're backwards compatible, but it still doesn't really give me any advantage if nobody else is using it. So what if rather than just using one VoIP gateway, there were hundred, scattered across every area code, and every network. Then when I call you, if I can't use my truly secure VoIP connection, instead it just routes you through one of hundreds of random gateways. Voila -- we both get protection from dragnet collection of metadata (the NSA just sees that someone called you through one of these many gateways, without knowing it's me) *even though* you don't use the system.<br />
<br />
Next, every time I call someone through this system and it falls back on the POTS gateway, it plays a message saying something like "This line is only partially secured; install XXXX app to get fully secured. Connecting..." Now every user who uses this thing is automatically advertising what it is to recipients. The more it's used, the more it grows. Indeed, you could also couple it with SMS such that the first time anybody calls a new number, it texts a link to that number explaining what it is and linking to an app download.<br />
<br />
Ok, so now we have a system that is backwards compatible, breaks the "chicken and the egg" dilemma by offering value "out of the box" even to a single user, and automatically promotes itself. But what about incoming calls? How can I get the benefit of anonymity, but still give you a number that you can reliably call to get me?<br />
<br />
This one is a lot harder. One approach would be to let me generate new phone numbers on the fly, such that I can give out different numbers to everyone and they all go back to me. Again, anybody who calls these numbers with POTS would get connected to me transparently via the VoIP gateway (and might hear the marketing message / receive the SMS), and anybody who calls inside the system gets me directly.<br />
<br />
A problem with this is there are only so many phone numbers, and they cost money. So a different approach might be to just maintain like a hundred numbers, each of which has an "extension". So I give you a number like (XXX) XXX-XXX x XXXX -- it's a bit of a pain to use extensions, but it gives the same effect.<br />
<br />
Then tie this with a Gmail plugin that auto-randomizes your phone number in emails you send out (so you enter your own phone number, and it provisions/randomizes before delivery), and maybe something that just provisions a bunch of random numbers and prints out business cards to make it easy to deliver.<br />
<br />
Oh, and all this could work for SMS as well.<br />
<br />
Anyway, something like this might allow individuals to "opt in" to a new secure platform, without needing to "opt out" from the real world.David Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com0San Francisco, CA, USA37.7749295 -122.4194155000000137.373502 -123.06486250000002 38.176356999999996 -121.77396850000001tag:blogger.com,1999:blog-8655656.post-90972052502906955432012-03-08T13:55:00.001-08:002012-03-08T13:55:33.856-08:00The Future of CopyrightMy entry to this contest:<p><a href="http://www.indiegogo.com/Future-of-Copyright">http://www.indiegogo.com/Future-of-Copyright</a><p>I predict the future of copyright will look a lot like today's war on<br>drugs: the only people supporting it will be people either profiting<br>from it or unaffected by it -- the people it purports to protect (not<br>to mention the people actually targeted) will have apathetic tolerance<br>for it, to futile resistance against it. Copyright will render the<br>entire content industry a wasteland of legal risk and weak offerings,<br>where all the top innovation and product development happens by<br>criminals. Piracy will become increasingly widespread and socially<br>acceptable, and future generations will just accept the absurd status<br>quo as "something previous generations did that we've just got to deal<br>with; thanks grandpa for fucking it up for everyone."David Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com0tag:blogger.com,1999:blog-8655656.post-51862855846377496272011-11-12T07:45:00.001-08:002011-11-12T07:45:42.000-08:00Fwd: Pho: How litigation only spurred on P2P file sharingAbout time somebody wrote a book on this!<p>-david<p>------<p><a href="http://www.itnews.com.au/News/279763,how-litigation-only-spurred-on-p2p-file-sharing.aspx">http://www.itnews.com.au/News/279763,how-litigation-only-spurred-on-p2p-file-sharing.aspx</a><p><br>How litigation only spurred on P2P file sharing<br>By Rebecca Giblin on Nov 11, 2011 12:30 PM (1 day 10 hours ago)<br>Filed under Telco/ISP<p>Analysis: Did the content industry lose the legal battle?<p>Do you remember back in 2001 when Napster shut down its servers? US<br>courts found Napster Inc was likely to be liable for the copyright<br>infringements of its users. Many of Napster's successors were also<br>shut down.<p>Aimster and its controversial CEO were forced into bankruptcy, the<br>highest court in the US strongly suggested that those behind Grokster<br>and Morpheus ought to be held liable for "inducing" their users to<br>infringe, and Kazaa's owners were held liable for authorisation by our<br>own Federal Court. Countless others fled the market in the wake of<br>these decisions with some, like the formerly defiant owners of<br>Bearshare and eDonkey, paying big settlements on the way out.<p>By most measures, this sounds like an emphatic victory for content<br>owners. But a funny thing happened in the wake of all of these<br>injunctions, shutdowns and settlements: the number of P2P file sharing<br>apps available in the market exploded.<p>By 2007, two years after the US Supreme Court decided Grokster, there<br>were more individual P2P applications available than there had ever<br>been before. The average number of users sharing files on file sharing<br>networks at any one time was nudging ten million and it was estimated<br>that P2P traffic had grown to comprise up to 90 percent of global<br>internet traffic. At that point content owners tacitly admitted<br>defeat, largely abandoning their long-time strategy of suing key P2P<br>software providers and diverting enforcement resources to alternatives<br>like graduated response or "three strikes" laws.<p>Why is it that, despite being ultimately successful in holding<br>individual P2P software providers liable for their users'<br>infringement, content owners' litigation strategy has failed to bring<br>about any meaningful reduction in the amount of P2P development and<br>infringement?<p>Physical vs digital<p>I would argue pre-P2P era law was based on a number of "physical<br>world" assumptions. That makes sense, since it evolved almost<br>exclusively with reference to physical world scenarios and<br>technologies. However, as it turns out, there is often a gap between<br>those assumptions and the realities of P2P software development.<p>Four such physical world assumptions are particularly notable in<br>explaining this phenomenon.<p>The first is that everybody is bound by physical world rules. Assuming<br>this rule had universal application, various secondary liability<br>principles evolved to make knowledge and control pre-requisites to<br>liability. But software has no such constraint. Programmers can write<br>software that will do things that are simply not possible or feasible<br>in the physical world. So once the Napster litigation made P2P<br>programmers aware of the rules about knowledge and control, they<br>simply coded Napster's successors to eliminate them – something no<br>provider of a physical world distribution technology ever managed to<br>do.<p>In response, the US Supreme Court in Grokster created a brand new<br>legal doctrine, called inducement, that did not rely on either<br>knowledge or control. That rule was aimed at capturing "bad actors" -<br>those P2P providers who aimed to profit from their users' infringement<br>and whose nefarious intent was demonstrated by "smoking guns" in their<br>marketing and other communications. But the inducement law failed to<br>appreciate some of the other differences that make the software world<br>special and thus led directly to the explosion in the number of P2P<br>technologies. In understanding why, three other physical world<br>assumptions come into play.<p>One is that it is expensive to create distribution technologies that<br>are capable of vast amounts of infringement. Of course in the physical<br>world, the creation of such technologies, like printing presses,<br>photocopiers, and VCRs required large investment. Research and<br>development, mass-manufacturing, marketing and delivery all require<br>massive amounts of cash. Thus, the law came to assume that the<br>creation of such technologies was expensive.<p>That led directly to the next assumption – that distribution<br>technologies are developed for profit. After all, nobody would be<br>investing those massive sums without some prospect of a return.<p>Finally comes the fourth assumption: that rational developers of<br>distribution technologies won't share their secrets with consumers or<br>competitors. Since they needed to recoup those massive investments,<br>they had no interest at all in giving them away.<p>All of these assumptions certainly can hold up in the software<br>development context. For example, those behind Kazaa spent a lot on<br>its development, squeezed out the maximum possible profit and kept its<br>source code a closely guarded secret. By creating a law that focused<br>on profits, business models and marketing, the Supreme Court succeeded<br>in shaking out Kazaa and its ilk from the market.<p>But the Court failed to appreciate that none of these things are<br>actually necessary to the creation of P2P file sharing software. It<br>can be so inexpensive to develop that some university programming<br>courses actually require students to make an app as part of an<br>assignment. When the software provider puts in such a small<br>investment, there's much less need to realise a profit. This, combined<br>with widespread norms within the software development community<br>encouraging sharing and collaboration, also leads to some individuals<br>making the source code of their software publicly available for others<br>to adapt and copy.<p>When the US Supreme Court created its new law holding P2P providers<br>liable where they "fostered" third party infringement, as evidenced by<br>such things as business models, marketing and internal communications,<br>the result was an enormous number of programmers choosing to create<br>new applications without any of those liability attracting elements.<br>In the absence of any evidence that they had set out to foster<br>infringement, they could not be liable for inducement, and having<br>coded out of knowledge and control they could not be held liable under<br>the pre-P2P law either.<p>The end result? The mismatch between the law's physical world<br>assumptions and the realities of the software world meant that the law<br>created to respond to the challenges of P2P file sharing led to the<br>opposite of the desired result: a massive increase in the availability<br>of P2P file sharing software. The failure of the law to recognise the<br>unique characteristics of software and software development meant the<br>abandonment of the litigation campaign against P2P providers was only<br>a matter of time.<p>Dr Rebecca Giblin is a member of Monash University's law faculty in<br>Melbourne. Her new book Code Wars tells the story of the decade-long<br>struggle between content owners and P2P software providers, tracing<br>the development of the fledgling technologies, the attempts to crush<br>them through litigation and legislation, and the remarkable ways in<br>which they evolved as their programmers sought ever more ingenious<br>means to remain one step ahead of the law. The book explains why the<br>litigation strategy against P2P providers was ultimately unsuccessful<br>in bringing about any meaningful reduction in the amount of P2P<br>development of infringement.<p>Visit <a href="http://codewarsbook.com">codewarsbook.com</a> where you can read the first chapter in full.<br>Physical copies can be ordered online from stores like Amazon and Book<br>Depository, and electronic copies are available via Google books at a<br>heavily discounted price.<p>Copyright © <a href="http://iTnews.com.au">iTnews.com.au</a> . All rights reserved.<p>---------------------------------------------------------------------<p>This is the Pho mailing list, hosted by griffinatonehousedotcom and <br>johnparresatgmaildotcom. Email one or both of us to unsubscribe, <br>subscribe or otherwise address any issues related to this list.David Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com0tag:blogger.com,1999:blog-8655656.post-60990352784936394582011-10-30T03:32:00.001-07:002011-10-30T03:32:45.057-07:00Our Place in EternityAfter an all-night reading binge, I finished re-reading Isaac Asimov's "<a href="http://en.wikipedia.org/wiki/The_End_of_Eternity">The End of Eternity</a>" -- one of my favorite books by my favorite author. I hadn't read it since, I don't even know, high school? In the nearly twenty years since it's always stuck with me. Like all of Asimov's work, I think he wraps an adequate story around some core, brilliant concepts.<br> <br> In this case it's exploring the consequences of humanity inventing, in essence, a "time elevator" -- step in at one year, and step out at any other. It can go backwards through time as far back as when it was created (in this case, around 2400), and forward as as far as when the sun becomes a supernova. The story centers around a group of people called The Eternals who manage and use the elevator, ostensibly for the purpose of enabling trade between times (your century have deforestation? import wood from the future!), but secretly also to tweak time for the greater good of humanity: something as simple as shifting a jar from one shelf to another could prevent disease, war, and the ever-present threat of nuclear apocalypse. Wrap in a mystery (why can't you get out of the elevator between the years 10M-10.5M, and why aren't there any people after 10.5M years?) a romance (between an Eternal and a Timer), and enough paradox-management to make <a href="http://www.imdb.com/title/tt0390384/">Primer</a> seem sensible, and you've got yourself a heck of a book.<br> <br> The first time I read it all those years ago, I was most interested in the ultimate consequences of a well-intentioned organization devoted to the ostensibly positive goal of mitigating the worst disasters of human history and ensuring trillions of lives achieve the happiest possible existence. (Read the book to learn how that all ends up.) <br> <br> But this time I was more interested in the minutiae of time alteration itself: the so called "ripple" of small changes having profound long-term consequences. This isn't a remotely new concept, but a few other recent developments made me think more on it.<br> <br> <br> Jumping to the present: I'm writing this from a balcony overlooking a river full of long, low boats in Hoi An, Vietnam. I'm here with my company for our annual month-long retreat, where we leave the real world behind for some foreign location far removed by space and time(zones) to work hard, get to know each other, and ultimately get a new perspective on all the things we take for granted -- in both our personal and professional lives. This is our <a href="http://www.facebook.com/media/set/?set=a.10150299209811025.338313.69088556024&type=3">fifth annual trip</a>, spanning this startup and the last, and it's always a very interesting experience.<br> <br> Most of it involves a of sitting around in cafes working on our laptops, but on weekends we typically head out on some adventure. Yesterday was such a day, and we rented moto-scooters and zipped over to the <a href="http://en.wikipedia.org/wiki/Marble_Mountains_%28Vietnam%29">Marble Mountains</a> -- five tall pillars of rock jutting out of an otherwise perfectly perfectly flat region. Atop the tallest of the five is an active Buddhist monastery, with tall pagodas and stunning vistas in all directions.<br> <br> But more interesting still were the caves worn into this rock over the millennia, each of which was repurposed into a different temple -- turning the entire mountain into a single huge temple, with awe-inspiring statues nestled deep within the earth. The improbable size of the underground caverns and their enormous carved-in-place statues was, at times, overwhelming. It's almost inconceivable to imagine the manpower required to first haul the materials for such a monastery up the near-vertical mountain trail, by hand, and then continue an equal distance back down into the earth to build within the caves. As a foreigner who doesn't share the religion (or in my case, any religion), I couldn't help but wonder: Who would do such a thing, and why?<br> <br> The "who" part of that is of of course clear: the large number of Buddhists in the region built the temple at enormous expense over a tremendous period of time. But the "why" is what captured my attention.<br> <br> Why spend so much energy on such a magnificent creation, in a region that clearly could have benefited by that energy being spent elsewhere?<br> <br> Now again, there are a host of obvious answers, each of which plays a part. For one, there's the allure of earning favor from the gods. When trying to influence your fate in this life (or position in the next), there are obvious advantages to participating in such an exercise.<br> <br> And of course there's the political and religious order that depends on the physical show of strength such imposing structures create. If we can cause this to happen in this world, so the reasoning goes, imagine what we can do in the next.<br> <br> But I think these are only symptoms of something more fundamental and universal. After all, imposing structures aren't the exclusive work of theistic religions. Athiestic ancestor worship nearly always includes lavish shrines or temples to honor the dead -- the people least likely to benefit from the attention. And this isn't just a religious desire. Nations, businesses, and even individuals invest in physical structures rich in symbolism: the national monuments, the stone facades of banks, or even a marble headstone to carry your name forward into history long after you're gone.<br> <br> What is it with the universal desire to transfigure the temporary into the permanent using tangible symbology? And why is the primary medium used almost always stone?<br> <br> And this brings me back to the book: I think there's a strong desire in nearly all people to create a "ripple in time", as big as they can. It's almost literally like dropping a stone into a river: the bigger the stone, the bigger the ripple. After all, diamonds may last forever, but they tend to move without your permission once you turn your back. A giant stone statue has a bit more permanence, especially when hidden on top of a giant pillar of rock, down in a deep cave, elevated on a tall platform out of reach, and physically larger than all openings. *That's* forever. <br> <br> So I could take this thought experiment in a few directions. One would be to challenge whether stone is the best medium for creating a splash. Other common ones include DNA, religion, ideology, myth, lore, legend, teaching, post-dated letters, memoirs, commissioned artwork, family heirlooms, etc. You can't help but create a ripple of some sort, or avoid being a product of the ripples that came before you. But you *can* take action in your life to maximize the extent of your ripple.<br> <br> (And there are high tech solutions like Kiva.org enabling micro-loans that re-invest to people in need when repaid. This has the effect of identifying people with entrepreneurial spirit, giving them the capital to grow their business and improve their standing in the world, enabling them to spread the entrepreneurial spirit through all the same ways everyone else has -- but with more means at their disposal. A single Kiva investment could be re-invested every 6 months for 50 years (based on a 2% default rate), meaning a single investment can help a hundred people over a major fraction of your lifetime. *That's* a ripple. -- Thanks to Matt McNamara for pointing this out to me!)<br> <br> I could also question why we have this innate need to make a splash, and whether it's universal or limited to a subset, whether this desire to make a splash can itself be taught (making the biggest ripple of all), or even whether that's something anyone might want to do.<br> <br> And I'm sure there are a dozen other interesting directions. But the direction I want to go with is to expand on an idea I wrote about previously, regarding the relationships between consciousness and tools.<br> <br> Now, you can read all about it in a <a href="http://blog.quinthar.com/2009/05/building-skynet.html">rambling essay</a> even longer than this one. But in short: I feel what separates humans from all other creatures is our exceptional ability to invent and use tools. (Yes, other animals do this too, but I think it's safe to say we're the best.) Furthermore, it's my belief that using a tool doesn't merely extend your reach, it physically -- in the most literal sense -- extends *you*. When you hold a hammer in your hand, the hammer is every bit as much a part of you as your hand, your spleen, or any other tool. We are in fact nothing but a collection of tools, all under some miraculous and ambiguous and sort of "conscious" control.<br> <br> Building on this notion, with tools being literal extensions of yourself, what are the ideas you have, the books you write, or the stone statues you build -- other than more tools? Sure, like any tools they're not all equally effective at achieving whatever intent you set the tool upon. But the right tools, maintained in the right context, might continue to be effective even after your body dies.<br> <br> And if the tools are literally a part of you, and if those tools continue to achieve your desired effects long after you die, did you really die at all?<br> <br> So long as there's some part of you -- some tool of you that's still functioning -- you're still alive. And if "life" is measured as the scope of your tools, is it possible that you might grow *more* alive over time?<br> <br> After all, the Buddha was just one guy in his day. But now he is a vast organization of billions of people. Maybe the secret to eternal life isn't through the supernatural, ascending to Heaven or Nirvana. Maybe it's just leaving a part of you -- the best part of you -- behind as your body degrades, such that it can grow eternally, freed from its confines?<br> <br> David Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com0tag:blogger.com,1999:blog-8655656.post-41812673480885688852011-04-08T11:07:00.000-07:002011-04-08T13:49:55.086-07:00Net Neutrality and In Flight WirelessI'm on a Delta flight equipped with GoGo in-flight wireless, and they <br>have an interesting campaign going on: free Twitter for all. It's a <br>pretty slick campaign, but I think it raises interesting net neutrality <br>issues because, in essence, Twitter is paying for preferred access.<p>Personally, I'm ok with it: I don't have any problem with an internet <br>carrier creating a "fast lane" that either side of the connection can <br>pay extra to use, so long as the lane is made equally available to all <br>comers, on the same terms.<p>That's not to say that all advertisers are required to accept <br>advertisements from all organizations -- I'm not excited about it, but I <br>wouldn't outlaw GoGo from accepting an ad for the Catholic Church on the <br>GoGo website while refusing an ad for atheism. As a publisher, GoGo can <br>choose what message to put on its own website, even if that message is <br>discriminatory.<p>But as a communication medium, GoGo shouldn't be allowed to grant free <br>access to websites hosted by the Catholic Church, while simultaneously <br>refusing the same deal to an atheist organization.<p>I understand it's a tricky and totally arbitrary line, but I think <br>content-discrimination should be legal (to enable free speech), while <br>communications-discrimination should be outlawed (to prevent restriction <br>of free speech).<p>I think too much of the NN debate is wrapped up in thinly-veiled <br>anti-corporate fearmongering (the little guys need to be protected from <br>the big guys!!). Even if it's a fine goal (and I don't think it is), it <br>doesn't seem to have any Constitutional or free/fair-market basis that I <br>can see.<p>Net neutrality shouldn't be about mandating equal performance, but equal <br>opportunity.<p>I'm curious what you think?<p>-davidDavid Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com0tag:blogger.com,1999:blog-8655656.post-90585403410081176992011-03-25T12:29:00.001-07:002011-03-25T12:29:17.922-07:00avg(exception) = nothingI'm on this mailing list where everybody is suddenly raving over this new book "<a href="http://www.amazon.com/Information-History-Theory-Flood/dp/0375423729/ref=sr_1_1?s=books&ie=UTF8&qid=1301079449&sr=1-1">The Information</a>". Amazon describes it as:<br> <br> <span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: 'Times New Roman'; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; font-size: medium;"><span class="Apple-style-span" style="font-family: verdana,arial,helvetica,sans-serif; font-size: small;">In a sense,<span class="Apple-converted-space"> </span><em>The Information</em><span class="Apple-converted-space"> </span>is a book about everything, from words themselves to talking drums, writing and lexicography, early attempts at an analytical engine, the telegraph and telephone, ENIAC, and the ubiquitous computers that followed. But that's just the "History." The "Theory" focuses on such 20th-century notables as Claude Shannon, Norbert Wiener, Alan Turing, and others who worked on coding, decoding, and re-coding both the meaning and the myriad messages transmitted via the media of their times. In the "Flood," Gleick explains genetics as biology's mechanism for informational exchange--Is a chicken just an egg's way of making another egg?--and discusses self-replicating memes (ideas as different as earworms and racism) as information's own evolving meta-life forms. Along the way, readers learn about music and quantum mechanics, why forgetting takes work, the meaning of an "interesting number," and why "[t]he bit is the ultimate unsplittable particle." What results is a visceral sense of information's contemporary precedence as a way of understanding the world, a physical/symbolic palimpsest of self-propelled exchange, the universe itself as the ultimate analytical engine. If Borges's "Library of Babel" is literature's iconic cautionary tale about the extreme of informational overload, Gleick sees the opposite, the world as an endlessly unfolding opportunity in which "creatures of the information" may just recognize themselves. --<em>Jason Kirk</em></span></span><br> <br> I don't know about you, but I can't piece together anything meaningful other than "Wow wow wow!!!!!"<br> <br> I'm really curious to hear if anybody who reads the book actually changes their opinion on anything as a result. I fear a lot of these books just have "something for everybody" such that you walk away feeling stronger in your belief no matter what that belief is. Sorta like MSG: it makes everything taste better, without having any flavor by itself. I'd love to hear somebody say "I've held this passionate belief my entire life, but as a result of reading this book I've changed my mind."<br> <br> <br> Somewhat related, I spoke at <a href="http://impact.kynetx.com/">a conference</a> recently, and the other presenters had these really incredible, well-researched, inspiring presentations. But I realized afterwards that a major problem with so many of these broad trend analyzes is they lack statistical relevance.<br> <br> For example, I find everybody talks about Twitter, Facebook, Google, and a half-dozen mega names -- and then draws inferences based on them. But that's equivalent to "averaging the exceptions", which just isn't a valid technique: the problem with outliers is they're *outliers* and by definition defy the baseline trends. They are too few and too different to be summarized in any meaningful way.<br> <br> Rather, I think these business-fad, pop-psychology, averaging-the-exception techniques just create hysteria and excitement where perhaps none is really warranted. Even if they're 100% "accurate", they're so incredibly imprecise as to be non-actionable. Said another way, even if you're totally right on predicting the wave, if you can't say with any certainty the time and magnitude when it will hit, it's not worth getting excited about.<br> <br> Don't get me wrong, hysteria and excitement are great ways to sell books or promote products. But as the people being sold and promoted *to*, it's in our interests to take these fantastic claims -- each of which seems increasingly fantastic with increasing frequency -- with a corresponding amount of skepticism and composure.<br> <br> <br> David Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com0tag:blogger.com,1999:blog-8655656.post-30968566422894605322011-03-22T20:20:00.001-07:002011-03-22T20:20:53.237-07:00iPad is a HandbagI'm here at the <a href="http://impact.kynetx.com/">Kynetx Impact</a> conference (come see me talk tomorrow at 11am!) learning about the "live web" through a series of keynotes. One of those keynotes will be moderated by Robert Scoble, and he happens to be sitting 5' to my left as I type these words. A few minutes ago I was labeled a "curmudgeon" (I didn't know that word was used anymore! but I managed to spell it right on first shot, so go me) for being an iPad skeptic. Robert took it upon himself to explain to me why the iPad is so incredible... and alas, it didn't take. But while he was trying, I think I learned *why* I'm an iPad skeptic: <b>because it's primarily a fashion accessory</b>, and I'm not fashionable.<br> <br> Now that's a bold statement. (The first one, not the second.) You might say "but it clearly has better workmanship than any competitor!" and "it does all sorts of genuinely helpful things!" And those statements are definitely true. But the same could be said of a haute couture handbag -- many of which cost vastly more than an iPad despite doing so much less.<br> <br> I've been toying with this notion for a while, but it really rung for me as Robert was trying to extol the virtues of the iPad -- clearly incredulous that I wasn't blown away. <br> <br> He brought up an app that shows a ton of videos in a huge virtual wall: an impressive work that looks super cool for browsing random videos. But I never do that; I probably look at a video sent to me by some friend maybe once a week, probably less. I'd never ever sit down and just randomly browse videos.<br> <br> Then he brought up Wolfram Alpha, showing the periodic table in an amazingly gorgeous, exquisite way. But I haven't needed a periodic table since high school.<br> <br> Then there was the cool news reader, this neat app for learning fiddles, etc. All of them are really neat, fantastic executions of their concept. Executions that simply couldn't be done on any other device -- executions that are made *possible* by the iPad.<br> <br> But their incredible executions of concepts that range from mildly to totally uninteresting. Given that, I just couldn't get excited about them, and that was clearly not the reaction he intended.<br> <br> At this point we highlighted that I'm incredibly far off the edge when it comes to my habits. I don't watch TV, I don't have a car, I work more or less continuously, and when I'm not on my <a href="http://www.engadget.com/2010/05/10/sony-vaio-p-series-review/">absurdly-small laptop</a> I'm drinking wine with my wife and walking my <a href="http://i.min.us/imnvbG.jpg">beagle</a>. I carry a Palm Pre (which replaced my <a href="http://en.wikipedia.org/wiki/T-Mobile_Sidekick">Sidekick</a>), I use Verizon Broadband (and <a href="http://en.wikipedia.org/wiki/Ricochet_%28internet_service%29">Ricochet</a> back in the day), etc, etc. He said "you make me look mainstream".<br> <br> Given all that, it's possible that I'm just so overworked and socially deficient that I simply cannot conceive of this value that is universally recognized by everyone else. It's possible.<br> <br> But I don't buy it. I think a more simple explanation is that I'm simply not fashionable.<br> <br> I think when most people see an iPad, they see this incredible world of possibilities -- and they want to participate in that world, even if <br> they don't personally use those possibilities in any meaningful way (or even if many of those possibilities don't actually exist yet). And I actually think that feeling of participation is akin or even equivalent to fashion. <br> <br> For example, Robert said Android wouldn't compete with iPhone until it had 10,000 *good* apps. But then he acknowledged that virtually everyone is always playing Angry Birds, or one of a tiny set of other apps. So I don't think the 10K app collection is important because people actually use those apps. I think it's necessary to create this image of endless possibility -- without that, the suspension of disbelief that's so critical to fashion just isn't there.<br> <br> Similar to fashionable clothing. A common theme is they always use the best materials, the highest quality stitching, the most exotic product placements and high-class endorsements, etc. I think all of these are necessary to create this image of supreme quality that justifies a 10x purchase price (or 10x brand loyalty) despite only being marginally better in any measurable way.<br> <br> Indeed, when I look back on my extreme product choices in the past, they actually *were* the best. I was doing email and browsing real webpages on my phone in 2002. I had wireless broadband in 2000. Compared to any Mac laptop, mine has a longer battery life, higher resolution screen, a smaller form factor, and built-in Verizon Broadband, etc. They were genuinely better than the other options at the time, but those options just weren't fashionable.<br> <br> But my point isn't to tout my awesomeness (though I could do that all day). Nor is my point to say the iPad isn't awesome (it is), or that tablets aren't superior to laptops for certain use cases (they are, though in far fewer cases than is usually claimed). <br> <br> Rather, I'm saying the iPad -- like any fashion accessory -- isn't nearly awesome as people say it is, and most of its differentiating value over other tablets is simply the strength of Apple's brand in telling a story of infinite possibilities, most of which don't actually matter, and many of which don't yet exist. <br> <br> David Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com8tag:blogger.com,1999:blog-8655656.post-47449147257102663122011-03-21T13:08:00.001-07:002011-03-21T13:09:38.137-07:00Google testing new amazing knowledge feature?I haven't seen this mentioned anywhere, but see screenshot below. I was curious when PayCycle was founded, so I searched "paycycle founded". Google apparently saw enough similarity in the search results that rather than just giving me the links, it gave me <b class="moz-txt-star"><span class="moz-txt-tag">*</span>the answer<span class="moz-txt-tag">*</span></b>. Especially interesting because not all of the answers were right (eg, the second search result is clearly wrong). Pretty amazing! <br /> <br /> <img alt="" src="http://i.min.us/ilZNtA.png" height="370" width="931"><br /> <br /> -david <br /> Founder and CEO of Expensify <br /> Follow us at <a class="moz-txt-link-freetext" href="http://twitter.com/expensify">http://twitter.com/expensify</a>David Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com1tag:blogger.com,1999:blog-8655656.post-90373872585020537242011-02-27T13:28:00.001-08:002011-02-27T13:38:23.335-08:00McDonalds isn't the problem; we are.<div>Everybody knows obesity is a problem, and that it's inflating medical costs that are gradually bankrupting our nation. But I think most people have a misguided sense that obesity is the result of fast-food using poor-quality ingredients and somehow tricking people into eating them. For example, I saw this article on BoingBoing talking about the uproar over the <a href="http://www.boingboing.net/2011/02/25/mcdonalds-oatmeal-ha.html">high calorie count in McDonald's new oatmeal</a>.</div> <div><br />Basically, it has "as much sugar as a Snicker's bar and as many calories as a hamburger". That sounds really alarming, but it made me wonder: how many calories does oatmeal normally have? What could McDonald's have possibly done to take something good and make it bad? So I did some research on oatmeal, only to eventually find that the BoingBoing commentators had done a lot more.<br /> <br />To make a long story short, the McDonald's oatmeal is totally fine. The oatmeal itself is mostly normal, and most of "extra" calories really come from them adding a bunch of dried fruit (which is hardly an atrocity) and adding brown sugar and cream by default (which is commonly done at home anyway). So... false alarm.<br /> <br />Again and again I think people overreact when it comes to the "quality" of fast food. Yes it's made fast and in high volume, but even with the freshest possible ingredients on hand I think the results would come out looking, tasting, and nourishing about the same. For example, In-N-Out arguably uses the freshest ingredients of any fast-food burger joint, and to compare:<br /> <br /><a href="http://nutrition.mcdonalds.com/nutritionexchange/itemDetailInfo.do?itemID=1">McDonald's hamburger</a> = 100g serving size, 250 calories, 9g fat<br /><a href="http://www.in-n-out.com/nutritional_info.asp">In-N-Out hamburger</a> = 243g serving size, 310 calories, 10g fat<br /> <br />(Incidentally, the standard In-N-Out burger comes with a spread that adds another 80 calories and 9g of fat. But I'm going with the mustard/ketchup option to compare more equally to McDonald's.)<br /><br />So the McDonald's burger has 2.5 calories/g, while the In-N-Out burger has only 1.3 calories/g. But both have about the same fat. What gives? My sense is the difference has nothing to do with the quality of the ingredients, and everything to do with In-N-Out putting heavy, water-filled veggies (lettuce, onion, tomato) on while McDonalds doesn't. I don't have the data in front of me, but I bet if you took all the veggies off the In-N-Out burger (or added an equal amount of veggies to the McDonald's burger) -- basically assembling them the same way -- you'd get largely the same results.<br /> <br />In other words, both use more or less the same quality ingredients, with essentially the same nutrition, despite McDonalds being demonized as the culinary antichrist while In-N-Out being some kind of organic savior.<br /> <br />In my opinion, the problem with McDonald's (or any other fast food chain) isn't that their food is so much higher calorie than if you were to fix it yourself. Rather, the problem is they cater to a customer base who is actively looking for high-calorie, high-fat food. Said another way, given a fully-stocked kitchen (and the willpower and expertise to actually cook), I wager most people would basically fix something as bad or worse than McDonald's, intentionally.<div> <br /></div><div>This is somewhat reinforced by <a href="http://www.telegraph.co.uk/health/healthnews/8325807/Calorie-labelling-has-no-effect-on-food-choices.html">this study</a> that suggests that NY's "label the calories as big as the price" plan is failing to produce results. I'll admit, I thought the plan was a good one, and I'm disappointed it didn't work. This suggests people know they're eating crap food (even if composed of reasonable-quality ingredients), but simply don't care.<br /> <br /></div><div><br />So where am I going with all this? I think the solution can't just demonize the quality of fast food ingredients (because they're fine) or emphasize how many calories people are buying (because they don't care). And it's not enough to highlight the long-term effects of those decisions; those are already pretty apparent and non-motivational.<br /> <br />Rather, we need some way to identify people who are on a bad long-term path and create short-term consequences. And by "we need" I mean "given that <a href="http://www.kpcb.com/usainc/">our country is being bankrupt by vast medical insurance programs</a> with out-of-control cost increases driven by health epidemics such as obesity, taxpayers should demand" that something be done to prevent people from taking actions that leave us on the hook for massive medical bills down the road.<br /> <br />Similar to how people with good driving records and safe-driving courses get lower insurance premiums, I think we should do the same for Medicare/Medicaid. Create programs where people can earn better care by making healthy choices. Granted, healthy people need less medical care so it doesn't make sense to give them *more* of it as a reward for needing *less* of it. But what if healthy people got tax credits and prioritized non-emergency care. Shorter waits, nicer rooms, more choice. Everybody still gets the same quality of medical attention (for better or worse), but people who actively maintain healthy lifestyles are rewarded with status, convenience, and comfort.<br /> <br />Furthermore -- and this is the most important point -- it should be made very clear to you which "service tier" you're in at all times, creating an *immediate* positive consequence for healthy actions that normally only have long-term effects. So everybody who does nothing is lumped into the "standard" tier; you needn't do anything special. But you should be constantly encouraged to upgrade to the "premium" tier by just demonstrating healthy decisions. How exactly that is done is obviously a big question, but some ideas:</div> <div><br />- Get credit for healthy-eating, healthy-lifestyle training courses<br />- Demonstrate participation in preventative care programs<br />- Get regular checkups to certify you haven't been abusing drugs<br />- Wear an <a href="http://www.technologyreview.com/biomedicine/22501/">electronic patch that measures caloric intake and expenditure</a><br /> - Join a gym and hire a certified trainer who reports activity to your doctor<br /><br />And so on. Every problem has a ton of complications, don't get me wrong. And it'll be a horribly political process to decide what's "healthy". But perhaps something like this can start to gradually steer us in the right direction?</div> <div><br /></div><div><br /></div><div>Admittedly, that won't be enough. Not even remotely close to what's needed to actually get things under control. But it might be a step in the right direction of preparing people to resume individual accountability for their health given we probably have little choice but to vastly scale back coverage (perhaps starting with reducing end-of-life care, which is <a href="http://neurologicalcorrelates.com/wordpress/2008/03/12/about-30-of-medicare-is-spent-on-end-of-life-care-what-should-we-do-about-it/">estimated to take roughly 30% of Medicare's budget</a>), followed by probable rationing of key medical resources. (Read here for a <a href="http://drscoundrels.com/?p=3516">hyperbolic freakout session</a> about kidney rationing, which obscures a few good ideas under a heap of total garbage.)</div> <div><br /></div><div>Ultimately, I'm all for reducing government involvement in a lot of things. But it will mean *reducing*, not eliminating. I think we should provide a *minimum* level of universal healthcare, recognizing that it's simply not possible to give maximum care to everybody. And we should eliminate barriers that prevent private insurance health plans from operating at maximum competitive effectiveness.</div> <div><br /></div><div>At the end of the day, very expensive or end-of-life treatment is a luxury for the rich, just like helicopters and fast cars. Whether we like it or not, that's just the way it is. But like helicopters and fast cars, they're terrible investments on which only the rich should waste their money. Instead, we should focus on expanding coverage of inexpensive, early-life care to everybody because it's an investment in society that's returns dividends to us all. And that's what the government is there to help us do.</div> <div><br /></div></div><div>-david</div>David Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com7tag:blogger.com,1999:blog-8655656.post-34954406530635249412011-02-05T10:48:00.001-08:002011-02-05T10:48:58.224-08:00Egypt's Internet Blackout and How to Build a Decentralized TwitterSo Egypt has its internet back, but I still can't figure out precisely <br>what was gone when it was gone. Can you help? So far as I can determine:<p>- Cellular service was only shut off in regions (eg, at the sites of the <br>major protests) while left on elsewhere in the country.<p>- Landlines continued functioning everywhere.<p>- I've seen no sign that domestic internet was affected. For example, <br>it's possible that all homes and businesses still had live network <br>connections that simply weren't resolving DNS, or perhaps *were* <br>resolving DNS internally. It's even possible that local DNS caches were <br>resolving completely normally -- even for international domains -- <br>except there were no routes to the IP addresses to which they resolved.<p>- Indeed, even if all ISPs turned off all broadband everywhere, there <br>would still be large pockets of functioning LANs (universities, housing <br>complexes, hotels, etc).<p>- The consequences of total domestic internet blackout are very severe. <br> I don't see any sign that government and critical services run their <br>own network (though the military might), nor any sign that the internet <br>was selectively disabled for only individuals and businesses while <br>sparing hospitals, police stations, power plants, etc. Furthermore, I <br>haven't heard that any critical services lost domestic internet or <br>telephone access, even though I imagine that would be a very interesting <br>story if true.<p>I think understanding what actually happened is important such that we <br>can plan and act in a way that is optimized for the real world, rather <br>than a (potentially) unreasonable worst-case scenario that never <br>actually occurs. I'd love your help in fact-checking the above <br>assumptions by providing evidence (links) to the contrary.<p><p>Regardless, none of this really changes my core thesis, which is that <br>whatever solution built must:<p>- Somehow become very popular and widely deployed *before* the event, <br>requiring substantial "added value" even when the internet is accessible.<p>- Define "added value" in terms that the average person cares about, <br>which is *not* anonymity, security, privacy, etc. Rather, it needs to <br>be speed, convenience, reliability, and so on.<p>- Take an approach of "adding to" rather than "replacing" whatever <br>more-popular alternatives are already in place (twitter, aim, <br>bittorrent, skype, etc) so as to ensure users sacrifice nothing by using it.<p>- Take best, simultaneous advantage of whatever resources are available, <br>at all times. If there is BlueTooth, use it. If there is a functioning <br>LAN, use it. If there is a functioning sitewide/domestic/international <br>WAN, use it. And so on.<p>- Anticipate the imminent failure of any of these methods at any time by <br>proactively establishing fallbacks (eg, a DHT in case DNS fails, gossip <br>in case the DHT fails, sneakernet in case wireless fails, etc.).<p>- Require no change in user behavior when one or more methods fail. So <br>the interface used to tweet, fileshare, make a call, etc -- all these <br>need to work the same for the user (to the greatest possible degree) <br>irrespective of what transport is used.<p>- Work on standard, unaltered consumer hardware (no custom firmware, mod <br>kits, jailbreaks, etc) with standard installation methods (app stores, <br>web, etc).<p>- Be incredibly easy for people to use who aren't tech-savvy. This <br>means spending 10x more time testing and refining the usability of the <br>system than actually developing sexy esoteric features.<p><p>I really do think this is a relatively easy thing to build (at least, in <br>a minimal form), using existing hardware and proven algorithms. I'd <br>suggest something like:<p>1) Start with an open-source twitter application. Google suggests this <br>one, though I haven't personally used it:<br> <a href="http://getbuzzbird.com/bb/">http://getbuzzbird.com/bb/</a><p>2) Add a central server, just to make it really easy for nodes to <br>communicate directly. (We'll replace this with a NAT-penetrating mesh <br>*after* the much more difficult task of getting this popular and widely <br>deployed.)<p>3) When you start up, connect to this central server. Furthermore, <br>whenever you see a tweet from anyone else using this client, "subscribe" <br>to that user via this central server. (Eventually you'd establish a <br>direct connection here, but we'll deal with that later.)<p>4) Every time you tweet, also post your tweet to this central server, <br>which rebroadcasts it in realtime to everyone subscribed to you. Voila: <br>we've just built an overlay on top of twitter, without the user even <br>knowing. All they will know is that tweets from other BuzzBird users <br>for some reason appear instantly. And the next time twitter goes down, <br>all tweets between buzzbird users will continue functioning as normal.<p>5) Then start layering features on top of this, focused on making a <br>twitter client that is legitimately the best -- irrespective of the <br>secret overlay.<p>6) For example, add a photo tweeting service. Publicly it'll use <br>twitpic or instagram or whatever, so all all other users will see your <br>photos just fine. But buzzbird users will broadcast the photo via this <br>central server, faster and more reliably than the other services, as <br>well as locally cached for offline viewing. Repeat for video, files, <br>phone calls, etc.<p>7) At some point when you've established that people actually like and <br>use buzzbird with its very simple and fast central server, THEN start <br>thinking about P2P. (Seriously, do NOT think about P2P before then as <br>it'll only slow you down and ensure your project fails.)<p>8) To start, keep the central server and just use it as a rendezvous <br>service for NAT penetration. So still centrally managed, but with <br>direct P2P connections. Then when you come online, you immediately try <br>to establish NAT-penetrated direct connections with everyone you're <br>following. This of course immediately presents you with challenges: do <br>you need to connect to *everyone* you follow? If only some, which? Take <br>these problems one at a time knowing you can always fall back on the <br>central server until you perfect it. In other words, the goal is to <br>remove the central server, but you can take baby steps there by weening <br>yourself off of it.<p>9) Similarly, add BlueTooth, ad-hoc wifi, USB hard drive sync (aka <br>"sneakernet"), etc. These would all be presented to the users in terms <br>of real world benefit ("Keep chatting while on the airplane!" "Share <br>huge files with a USB flash drive!" and so on.), while simultaneously <br>refining the tools that they'll use during a partial or total internet <br>blackout.<p>10) Eventually when you've figured out how to move all functions off the <br>central server -- the nodes start up, establish direct connections to <br>some relevant subset of each other, build a DHT or mesh network, nodes <br>that can relay for nodes that can't, etc -- the last function will be <br>"how does a newly-installed node find its first peer?" This is called <br>the "bootstrapping" problem, and is typically done with a central <br>server. But it needn't be done with *your* server. Just use twitter <br>for this: every time you start, re-watermark your profile image with <br>your latest IP address (or perhaps put it into your twitter signature <br>line, or location, or something). This way the moment you do your first <br>tweet, everybody who sees it will try to contact you (or some subset, so <br>as to not overload you). Then you can turn off your own central service <br>and just use twitters.<p>11) When the "dark days" come and twitter goes offline, your nodes won't <br>even notice. They'll continue to establish their DHT with whatever <br>subset of the network is interconnected, relay for each other, etc. If <br>the internet is totally gone, your users will use bluetooth, wifi, <br>sneakernets. They'll be ready because you *trained* them to survive on <br>their own *before* they needed to, rather than just handing them a knife <br>assuming they'll know how to use it when the time comes.<p><p>Really, the biggest challenge in all of this is whoever gets to step (6) <br>will immediately be acquired by Twitter for an enormous sum of money. <br>But hopefully that person will, with their new-found wealth, continue on <br>to (7) and beyond.<p>This is a doable thing. One person motivated person could do most if <br>not all of this. Is that person you?<p>-davidDavid Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com3tag:blogger.com,1999:blog-8655656.post-88539148279864429822011-01-31T15:35:00.000-08:002011-02-01T23:54:58.190-08:00What we should build for the Egyptian (and other) protestersEgypt appears to have cut all internet connectivity with the rest of the <br />world in an attempt to quell its use in organizing protests. The only <br />reason this makes any sense is if the tools used to organize the <br />protests (Twitter, Facebook, Gmail, etc) are hosted outside Egypt.<p>To this you might say "Let's just host protest-organizing tools on <br />servers inside protest-likely nations in anticipation of them using this <br />strategy again." But that won't work because odds are the government <br />would just seize all protest-organizing servers within their borders.<p>So the only protest-tools that will continue to work reliably are those <br />that continue to work without access to the outside world, without <br />relying on locally-hosted servers, and *without even relying on the <br />internet at all*. It's a tall order, but here's how I'd do it.<p>1) Recognize that this service needs to be used in the good days, such <br />that there is adequate distribution already in place when the bad days <br />happen. THIS IS THE HARDEST PART. I say this in all caps because this <br />is why no meaningful system like this exists today: the people most <br />likely to build it are too obsessed with esoteric technical problems <br />than solving the issues that actually matter in the real world. <br />Asymmetric, anonymized, mesh-distributed, onionskin-routed communication <br />doesn't mean anything if nobody uses it. So before even thinking about <br />the technology, we need to think how to make it relevant to users who <br />*aren't* protesting (yet).<p>2) At an absolute minimum, it needs to be no worse than then existing <br />alternatives. So if it's going to replicate Twitter, it needs to be at <br />*least* as good as Twitter, otherwise everybody will use the *real* <br />Twitter (until it's turned off by their local neighborhood dictator). <br />On way to be better than Twitter is to actually be better than Twitter. <br /> Good luck with that. Another way is to just make your tool post to <br />Twitter. I think that's a much better idea: if this tool (let's call it <br />"anoninet" just for kicks) offers some Twitter-like functionality, it <br />should be completely compatible with the real Twitter in the <br />99.99999999999% of situations where the real Twitter is actually <br />available. Same goes for Facebook, Flickr, etc.<p>3) Ok, so anoninet's primary value in "good times" is starting to take <br />shape: it's a one-stop-shop to post to all your social networks. So you <br />install this thing, type in all your passwords (You could store them <br />locally in some encrypted keychain decrypted by a master password, but <br />that's the sort of technomasturbation thinking that obscures real-world <br />requirements; in reality just store it unencrypted because those who <br />don't care don't care, and those who do should really just encrypt their <br />whole hard drive), then you can post status updates, photos, videos, and <br />everything will automatically go to the right place. Indeed, before you <br />even think about making this into some sort of resilient <br />protest-enabling tool, you should make this the best possible <br />social-network posting tool. (Because if it's not that, then nobody <br />will have it installed when they want it most.) I'd suggest emphasizing <br />how this thing works even with unreliable internet, essentially letting <br />you queue up everything locally and it does background uploading as the <br />network becomes available. Similarly, it downloads everything locally <br />for offline reading. Odds are your protest-likely environment has <br />shitty internet to start, so this feature will likely have immediate <br />value. Add in really good support for USB-connected devices (cameras, <br />videocams), and basically present it as the single best way to do social <br />networking in a nation with shitty internet.<p>4) Step 4 is to succeed with step (3). Don't even think of anything <br />else until you've done that. Seriously, it's a waste of your time and a <br />disservice to your users. (3) needs to be totally nailed and immensely <br />popular before anything else matters. I'd say something like 10% of <br />your target population needs to be using it before you consider continuing.<p>5) Once you've got huge distribution of your client-side <br />social-network-optimizer, then you can start to raise the bar. Because <br />it's targeted to environments that have expensive and/or unreliable <br />internet, P2P starts to sound interesting. Throw in a network-localized <br />DHT and build out a distribution network that "rides" on these other <br />networks. So every time they post to Twitter, Facebook, Flickr, <br />YouTube, or whatever -- they're also posting to anoninet. And when <br />another anoninet is reading your Twitter stream, somehow they detect <br />each other and rather than getting the data from Twitter (for example), <br />they get it directly via some localized P2P connection. Present this to <br />the user as faster, more reliable, and cheaper than getting it from the <br />*real* YouTube.<p>6) Quietly encrypt everything and tunnel over commonly-used ports. <br />Don't talk about this, just do it. Users don't care until they do, and <br />by then it's too late.<p>7) Ok, so at this point we have wide distribution of a very popular <br />social networking tool that uses a localized P2P mesh as an optimized <br />fallback to the major global tools. Its major advantage is it works <br />over networks that are slow, unreliable, or expensive. This'll save you <br />in the Egypt case; these users would continue using the tools they <br />already use, to talk to the people they already talk with, and <br />everything will continue functioning as normal. They won't be able to <br />talk with the rest of the world, but they *will* be able to talk amongst <br />themselves, which is the important thing. Furthermore, because it's all <br />P2P, there are no servers to seize, and because it's all encrypted over <br />common ports, it's indistinguishable from all other encrypted traffic.<p>8) However, if this had existed in Egypt, odds are Egypt would have just <br />shut down the internet, period. If a dictator is willing kill you, odds <br />are they wouldn't blink at turning off your email. So how to make this <br />work without internet? The answer is: make it incredibly easy to batch <br />and retransmit data like Fidonet back in the day. So when shit is <br />*really* going down, you whip out your favorite 4GB, 32GB, or 640GB USB <br />drive and just sync your local repository (remember how everything was <br />conveniently cached locally for fast offline access?) with the device. <br />Optimize it to sync the most popular content first, basically ensuring <br />that the most intersting/important message is also the most widely and <br />redundantly distributed.<p>9) Finally, this needs to spit out an installable copy of itself to <br />whatever removable media is available. This way when the shit starts to <br />*really* go down, as people realize the true value of this system it can <br />spread fast to the people who need it.<p>Voila. A tool that supports communication amongst protesters even in <br />the face of total internet blackout. Some other random thoughts:<p>- Ideally it'd piggyback on existing credentials. So when you install <br />this thing you don't need to think "I'm creating a new account". <br />Rather, you just install this thing, type in your Twitter username and <br />password, and whatever giant asymmetric keypair it creates internally is <br />just some nameless thing associated with that Twitter account. (And you <br />might have multiple.)<p>- This thing needs to broadcast itself via existing networks in a <br />totally transparent way, so if we're both users and I read your Twitter <br />stream, I should know you're also a user without you ever telling me. <br />The first way that comes to mind is this thing could watermark your <br />profile image with maybe a digital signature (or perhaps just jam it <br />into some sort of extra field in the image). Then when I follow you, my <br />client sees the watermark, reaches out to the DHT, sees that you're <br />signed in (or not), and establishes a NAT-tunneled P2P connection directly.<p>- Social networks are particularly good for this sort of architecture as <br />they map well to the "publish/subscribe" model. This works easily on a <br />P2P network (you register yourself with the DHT by name and <br />keyword/hashtag, and then when you post there everybody who is <br />"following" you or a particular hashtag gets your data), as well as <br />create an implicit "value" metric for use when synchronizing data in <br />"sneakernet mode" (publishers/hashtags with a high follower count are <br />assumed to be more valuable and thus beat out less-popular content).<p>- This sort of system actually isn't that useful to terrorists, <br />criminals, drug-dealers, and so on because it's designed for mass public <br />communication (not indvidual private communications). Granted, nothing <br />in this protects the individual from being targeted, but that's an <br />entirely different problem. (And I wager one that could be layered on <br />top of this in a straightforward manner.)<p>In all honesty, this isn't that hard a thing to build. One dude could <br />do it. I could personally do it, and know several others who could as <br />well. But I'm busy. Hopefully a better person than me with more time <br />on their hands will pick up on this and do what needs to be done. The <br />world will thank them for it, though its dictators won't.<p>-david<br />My blog (including this post) is at <a href="http://quinthar.com">http://quinthar.com</a><br />Follow me at <a href="http://twitter.com/quinthar">http://twitter.com/quinthar</a>David Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com0tag:blogger.com,1999:blog-8655656.post-54689263128260769222011-01-25T00:07:00.000-08:002011-01-25T00:08:04.024-08:00From the archive: David's Voluntary Payment PlanThis one is from 2008. I was asked something along the lines of "Well if <i>you're</i> so so smart, how would <i>you</i> fix the music industry?" Here's my answer:<br> <br> <a href="http://quinthar.com/DavidsVoluntaryPaymentPlan.html">http://quinthar.com/DavidsVoluntaryPaymentPlan.html</a><br> <br> <span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: 'Times New Roman'; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; font-size: medium;"> <p style="margin-bottom: 0.08in; direction: ltr; color: rgb(0, 0, 0); margin-top: 0.17in; page-break-after: avoid;" align="CENTER"><font size="5"><b>David's Voluntary Payment Plan</b></font></p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;" align="CENTER"><font style="font-size: 8pt;" size="1"><i><a href="http://quinthar.com/">David Barrett</a></i></font></p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;" align="CENTER"><font color="#000080"><u><a href="mailto:dbarrett@quinthar.com"><font style="font-size: 8pt;" size="1"><i>dbarrett@quinthar.com</i></font></a></u></font></p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;" align="CENTER"><font style="font-size: 8pt;" size="1"><i>2008/3/20</i></font></p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;" align="CENTER"><br> </p> <h1 class="western" style="margin-bottom: 0.08in; direction: ltr; color: rgb(0, 0, 0); text-decoration: underline; font-family: Helvetica,sans-serif; font-size: 16pt;">Abstract</h1> <p class="western" style="margin-bottom: 0.08in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">This plan recommends creating “music registrars” to authoritatively manage song metadata in a fashion similar to how domain registrars authoritatively do the same for domain names. Artists (or their representatives) upload songs to registrars, who in turn check their waveform fingerprints against a master database of all known songs. If the song has already been registered by another owner, a conflict resolution process is started. Otherwise, the song is transcoded to a MP3 and tagged with a variety of metadata (artist and song name, artist website, etc), including “payment protocols” that enable fans to support the artist in a standardized way. iPods and other MP3 players are gradually outfitted with integrated support for various payment protocols, as well as methods for receiving artist communication or learning of and purchasing artist merchandise, concert tickets, and so forth.</p> <h1 class="western" style="margin-bottom: 0.08in; direction: ltr; color: rgb(0, 0, 0); text-decoration: underline; font-family: Helvetica,sans-serif; font-size: 16pt;">I. Example of Operation</h1> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">First, here's a quick walkthrough of how the system would be used in common operation:</p> <h2 class="western" style="margin-bottom: 0.08in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 14pt; font-style: italic;">A. Adding a new song</h2> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">Alice, an independent musician, selects from one of several music registrars, creates a free account, uploads her track in the FLAC format, assigns it a name, optionally organizes it in one or more albums, and is done. The entire operation is free, takes less than 10 minutes, and requires no personal information beyond an email address.</p> <h2 class="western" style="margin-bottom: 0.08in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 14pt; font-style: italic;">B. Downloading a song</h2> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">Bob, a music aficionado, browses a variety of free music outlets for new songs. One of those locations has an active online community around indie music, and the forum is buzzing around a new musician, Alice. The forum links to a page where Alice's music can be downloaded -- he clicks the link, chooses the format and bitrate, and downloads the MP3 for free. Though the website allows low-quality 128Kbps versions of the song to be downloaded or streamed straight from the server, for cost reasons it only allows 256Kbps and FLAC versions to be downloaded via a P2P network. He's all about quality, so he whips out his favorite P2P application and downloads the FLAC.</p> <h2 class="western" style="margin-bottom: 0.08in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 14pt; font-style: italic;">C. Listening to a song.</h2> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">When the download completes, Bob copies the file several places -- his laptop, his home stereo, his iPod, his phone -- all of which support the completely standard, unprotected audio format.</p> <h2 class="western" style="margin-bottom: 0.08in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 14pt; font-style: italic;">D. Supporting Alice</h2> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">Bob decides that he really likes Alice's music and wants to see more of it get played. He has several ways to help that happen:</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><br> </p> <ul> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">One way is to go back to the website where he downloaded the music in the first place. There there's a small (but growing) forum where Alice fans discuss her music, links to other music by Alice, recommendations of other music by Alice, and so on. Furthermore, there's a quick note by Alice herself saying "Hi, I'm trying to raise $1000 to fund my next album, please help me out!" Bob sees she's up to $950 right now. He's got a few options of how to help. One is to just do a simple cash contribution, one is to help raise up to $1000 (at $950 so far) with the caveat that if she doesn't raise the full amount within a set timeframe, the money is given back. Another is a subscription of $1/mo that gets his name put on a list of True Fans. Yet another is to buy the last limited-edition autographed copy of Alice's first Vinyl album for $50. All of these options can be paid with PayPal or a credit card.<br> <br> </p> </li> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">Another way is to use a feature built into iTunes and his iPod to auto-support any any song he listens to more than 5 times, to the default (but adjustable) amount of $0.05/listen. Similarly, whenever he looks at the face of his iPod to remember who he's listening to, he sees Alice's message that she's trying to raise $1000 and is up to $950. Likewise, he sees there's one more copy of the limited edition vinyl available.</p> </li> </ul> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><br> </p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">Ultimately, he decides to go for the vinyl recommended by his iPod. He goes to iTunes, chooses "open musician's website", and buys the vinyl online.</p> <h2 class="western" style="margin-bottom: 0.08in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 14pt; font-style: italic;">E. Getting paid</h2> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">When Alice signed up, she had no idea her music would be such a hit. But her inbox is full of messages, donations, and all her vinyl copies (which she hasn't even made yet) have already been sold.</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><br> </p> <ul> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">Getting to work, she uploads the cover art design and asks her registrar to press the given number of vinyl records and FedEx to her for signing. When she sends them back, the company redistributes them to the customers who purchased them, and the money is deposited into her account.<br> <br> </p> </li> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">As for how to get her money, she has a couple options. The classic approach is to just give her direct deposit information and it's deposited via the ACH network (automated clearing house). Another is to give her PayPal information. She doesn't like any of those options, so she goes with a third option of just having a reloadable prepaid Visa card sent her way -- any money added to her account is instantly available for use at any merchant, or even to be withdrawn from any ATM.</p> </li> </ul> <h1 class="western" style="margin-bottom: 0.08in; direction: ltr; color: rgb(0, 0, 0); text-decoration: underline; font-family: Helvetica,sans-serif; font-size: 16pt;">II. Music Registrars</h1> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">Core to this plan is the notion of "music registrars". Like DNS registrars (from which this draws inspiration), there are many and all provide compatible functionality while competing aggressively on price and value-added services. Musicians are free at any time to sign up with any number of registrars, or move tracks between registrars at a later date. But each track ultimately maps back to a single registrar that manages (at least) standardized metadata operations around that track. In essence, a registrar provides<span class="Apple-converted-space"> </span><i>at least</i><span class="Apple-converted-space"> </span>the following:</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><br> </p> <ul> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>Account creation.</u><span class="Apple-converted-space"> </span>Generally with a username/password, though optionally with more secure mechanisms (multi-factor authentication, PKI, etc).<br> <br> </p> </li> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>FLAC storage.</u><span class="Apple-converted-space"> </span>For every track managed, permanently store a master FLAC version.<br> <br> </p> </li> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>Metadata hosting.</u><span class="Apple-converted-space"> </span>For a given track, host its authoritative name, artist, album, etc. (essentially, ID3 tags) in one or more languages.</p> </li> </ul> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><br> </p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">Though not strictly required, in general a registrar will offer a wide variety of additional services, including some subset of:</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><br> </p> <ul> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>Transcoding and hosting.</u><span class="Apple-converted-space"> </span>Generates a variety of file formats from the master FLAC, including MP3, Flash, etc. and hosts them on the web and P2P networks.<br> <br> </p> </li> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>Payment gateway.</u><span class="Apple-converted-space"> </span>Accepts payments from fans according to a variety of payment protocols and securely deposits into the artist's account.<br> <br> </p> </li> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>Fan management.</u><span class="Apple-converted-space"> </span>Forums, blogs, RSS feeds, and all the accouterments of web 2.0.<br> <br> </p> </li> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>eCommerce.</u><span class="Apple-converted-space"> </span>Anything ranging from a Yahoo Store-like checkout system to a CafePress-style product generation assistant.<br> <br> </p> </li> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">Recommendation engines, playlist management, webcasting radio stations, promotion services, gig management, tour assistance, discount music equipment, etc. Basically, each registrar will attempt to provide artists with a complete one-stop-shop of all things they could possibly need to be a happy, successful musician.</p> </li> </ul> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><br> </p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">A service exists that lets anybody look up the latest metadata on any track. (Typically you would just download the metadata straight from its registrar, but there would be a mechanism to determine who the registrar is -- if any -- for an unknown piece of music.) This service uses a combination of servers hosted by the registrars, as well as servers hosted by an independent organization that manages the registrars themselves. This organization is focused exclusively on the operation of enabling transfers of music between registrars, resolving disputes between registrars (and between users and registrars), and authoritatively stating which registrar is currently managing which track. This organization is funded through annual re-certification fees paid to the organization by registrars.</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><br> </p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">One operation that is particularly interesting is: how does this organization uniquely identify each track in order to guarantee that each is only being represented by a single registrar? The answer is by using waveform fingerprints. Each registrar holds onto the master FLAC for every song in its management. Upon adding a new song, it uploads a "fingerprint" of the song to the master organization, which then confirms no other song has the same signature. (If there is a conflict, the organization investigates and resolves it.) The organization will make the choice as to which signature function to use (and it needn't be perfect, it's just a tool in helping proactively identify and resolve conflicts), and it can at any point decide to use a new function by simply having all registrars re-fingerprint all FLACs with the new function. Again, the fingerprinting doesn't need to be (and won't be) perfect -- it's just a flag that triggers manual corrective action. The better the function, the less wasted work.</p> <h1 class="western" style="margin-bottom: 0.08in; direction: ltr; color: rgb(0, 0, 0); text-decoration: underline; font-family: Helvetica,sans-serif; font-size: 16pt;">III. MP3, ID3, and Metadata</h1> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">In general practice, a musician would upload a track's master FLAC to her music registrar, and the registrar would generate a series of MP3s that have all the ID3 tags correctly set. The musician could then do whatever she liked with those MP3s -- email them, post them to P2P networks, post them on forums, burn them to CDs, etc -- and the ID3 tags would just be carried along with them.</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><br> </p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">However, the metadata can be indexed, distributed, and used in any way, even outside of MP3s -- the same information can be downloaded from the registrar at any time.</p> <h1 class="western" style="margin-bottom: 0.08in; direction: ltr; color: rgb(0, 0, 0); text-decoration: underline; font-family: Helvetica,sans-serif; font-size: 16pt;">IV. Music Metadata and Player Support</h1> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">In general, the metadata associated with a particular song can be any arbitrary name/value pair that the owner sees fit to associate with the song. There are no strict requirements or limitations on what sort of metadata must be associated. Similarly, players can choose to support all, none, or any subset of the metadata contained within a file. Any metadata not understood should be simply ignored. Some types of metadata include:</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><br> </p> <ul> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>The standard ID3 tags</u>: The obvious metadata includes artist name, song name, album, genre, and everything else you typically see in MP3 players. Example:<br> <tt class="western" style="font-family: 'Nimbus Mono L','Courier New',monospace; font-size: 10pt;">Name: Before Today<br> Artist: Everything but the Girl<br> Album: Walking Wounded<br> Track: 1</tt><br> <br> </p> </li> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>Unique song GUID</u>: A globally unique identifier assigned by the registrar to this song. A given song would have the same GUID across all bitrates and encodings, for example, but different mixes of this song would have different GUIDs. In general, all MP3s with the same GUID should have the same waveform fingerprint; similarly, in general, no two tracks with different GUIDs should have the same waveform fingerprint. This GUID can be used by the player, website, or other service for whatever purpose it likes (it's handy to have a key by which to index the song). Example:<br> <tt class="western" style="font-family: 'Nimbus Mono L','Courier New',monospace; font-size: 10pt;">GUID: s8d9fgfud6s6d6f8ds8sys6s65</tt><br> <br> </p> </li> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>Metadata URL</u>: A new tag would be a HTTP URL from which the latest authoritative metadata can always be downloaded in some standard format (I'd propose JSON, others might argue XML, but the specific choice is TBD). Any player or service can download the latest metadata for this track at any time, possibly rewriting the MP3 itself with the new information. Example:<br> <tt class="western" style="font-family: 'Nimbus Mono L','Courier New',monospace; font-size: 10pt;">MetadataURL: <a class="moz-txt-link-freetext" href="http://mytunes.com/meta/s8d9fgfud6s6d6f8ds8sys6s65">http://mytunes.com/meta/s8d9fgfud6s6d6f8ds8sys6s65</a></tt></p> </li> </ul> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.5in;"><br> </p> <ul> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>Payment protocols</u>: A series of descriptions through which this artist can be automatically compensated according to some predetermined protocol. There will be many different payment protocols (and new ones all the time), some of which might include direct deposits into bank accounts, charging to phone bills, reverse charges to prepaid credit cards, PayPal transfers, eGold transfers, or whatever. It's likely each registrar would offer one or more of the most well-known payment protocols by default, but there is no restriction on somebody coming out with a new payment protocol and then associating it with their song. (More details on this below.) Example:<br> <tt class="western" style="font-family: 'Nimbus Mono L','Courier New',monospace; font-size: 10pt;">Payment: ach://<bankaccount>,<institution ID><br> Payment: paypal://<email address><br> Payment:<span class="Apple-converted-space"> </span></tt><font color="#000080"><u><a href="http://mytunes.com/s8d9fgfud6s6d6f8ds8sys6s65">http://mytunes.com/s8d9fgfud6s6d6f8ds8sys6s65</a></u></font><tt class="western" style="font-family: 'Nimbus Mono L','Courier New',monospace; font-size: 10pt;"><br> Payment: raise://amount=$1000&current=$950&by=2008/4/1</tt><br> <br> </p> </li> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>Hash</u>: Though there's no strict requirement that a given song be distributed universally as a binary-identical MP3 for each given bitrate, it's reasonable to assume that this convergence would occur. Thus a valid piece of metadata would be the hash of a given encoding, which can be used by the player to verify that the file hasn't been corrupted. Example:<br> <tt class="western" style="font-family: 'Nimbus Mono L','Courier New',monospace; font-size: 10pt;">Hash: MP3/256/SHA1(3da3f0afc0d772825c43e310fe34eacf0dea204b)</tt><br> <br> </p> </li> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>Message of the day</u>: A general message that the artist wants to associate with this song. Can be anything from a simple hello, a description of the song, a request for help, an advertisement, or anything. This could appear on the face of an MP3 player, or in a bubble on your desktop, or however the player feels fit to show it. Example:<br> <tt class="western" style="font-family: 'Nimbus Mono L','Courier New',monospace; font-size: 10pt;">MoTD: Only 1 copy left of my limited edition vinyl album, $50!<br> MoTD: Don't forget, I'm playing the Fillmore tonight at 8pm!</tt><br> <br> </p> </li> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>Lyrics</u>: The lyrics of the song itself could be easily included in the song, or perhaps a URL where the lyrics can be downloaded.<br> <br> </p> </li> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>Other songs by this artist / recommended by this artist</u>: Links to other songs by this artist. A player could be configured to poll this at some frequency to be automatically notified when new music by an artist becomes available.</p> </li> </ul> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><br> </p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">The important thing to take away is that metadata can contain anything, and registrars merely record and host it -- it might or might not have any awareness of what the various name/value pairs actually mean. You needn't ask anybody's permission or get the approval of any standards body to create new metadata: just add it to your song, and any player that doesn't expect it will ignore it.</p> <h1 class="western" style="margin-bottom: 0.08in; direction: ltr; color: rgb(0, 0, 0); text-decoration: underline; font-family: Helvetica,sans-serif; font-size: 16pt;">V. Artist Compensation via Player Integration</h1> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">The basis of this system is to enable fans who want to compensate artists whenever and wherever the mood strikes them, in whatever amount, for whatever reason they come up with. This is enabled through integration with the players themselves, as this reduces the latency between hearing the song, making the decision to support the artist, and actually conducting the transaction.</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><br> </p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">The specific method of the integration is up to the designer of the player or service. But some examples that could be applied to any general MP3 player include a "thumb's up button" where $0.50 is sent to the artist when pressed, or an "auto-tip" option where $0.05 is sent to the artist each time his song is played in entirety, etc. All of this would be opt-in and configurable by the user in regards to the amount being paid and the frequency of payment.</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><br> </p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">Similarly, metadata and players could generally conform to standard ways of advertising merchandise and concert tickets related to the music. Depending on the player's form factor, it could even provide basic storefronts, one-click additions of tour dates to Google Calendars, or whatever type of interaction the device feels is appropriate to facilitate between artists and fans (perhaps even with a commission for the transaction paid to the device manufacturer). Ultimately, this is left up to the artists, fans, and player manufacturers to decide – the music registrar just manages the metadata without being aware of what it means or how it's used.</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><br> </p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">As for how the payment would be technically conducted, this would depend on the payment protocol and would likely be decided by a period of competition ultimately leading to a few widely supported "de facto" standards. For example, a phone-integrated player might use a payment protocol that puts song contributions straight onto your phone bill. An iPod might keep an internal count of what payouts are left to be done, and then upload the transactions to an iTunes-integrated micropayment engine when synchronized. WinAMP might accumulate transactions until they exceed some threshold where paying the artist directly via PayPal makes sense. And so on. Payment providers will compete vigorously for adoption by players and registrars alike, but the ultimate decision for who to pay, how, and how much rests with the listener.</p> <h1 class="western" style="margin-bottom: 0.08in; direction: ltr; color: rgb(0, 0, 0); text-decoration: underline; font-family: Helvetica,sans-serif; font-size: 16pt;">VI. Conclusion</h1> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">In summary, the above proposal outlines a global framework where fans can voluntarily support fans through a competitive ecosystem of compatible service providers. The design separates functionality along clear layers of accountability and enables competition between multiple parties within the layers. The goal is to create a flexible, powerful system that enables a degree of innovation yet unseen in the music industry (at least, in the legal music industry). Much like the web and internet itself have transitioned from small, non-profit research projects into engines of global commerce, music -- both its creation and consumption -- has the capability to be a similarly innovative and powerful force. It just needs a framework that encourages it.</p> <h1 class="western" style="margin-bottom: 0.08in; direction: ltr; color: rgb(0, 0, 0); text-decoration: underline; font-family: Helvetica,sans-serif; font-size: 16pt;">VII. FAQ</h1> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;">Here are the questions I've heard asked on this list before, and some quick answers to each:</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><br> </p> <ol> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>What if nobody decides to pay?</u><br> The base assumption of the entire music industry is that music is valuable, and that fans actually do exist. If fans -- people who value art and wish to support their artists -- do not in fact exist, then this system won't create them.<br> <br> </p> </li> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>What if no music players decide to support payment options?</u><br> The system works best if the payment protocols are implemented in the players themselves. In the meantime, until these are widespread, music registrars can offer web-based gateways that help fans support artists using today's technology.<br> <br> </p> </li> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>What's to prevent me from uploading the Beatles as my own mine?<br> </u>The standard solution to this problem is to have a "sunrise period" where prominent trademark and copyright owners are given early access to submit their own songs to the database. The expectation is each of the labels would run its own "private" registrar to manage its songs, and thus they would simply upload a complete list of fingerprints for all their songs to the registrar-management agency. In the event anybody uploads one of the label's songs to a different registrar, a flag would be raised when the fingerprint conflicts with the existing database, and would be resolved through manual action.<br> <br> </p> </li> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>So... where's the big pool of money? Where's the sampling?<br> </u>That's right, this system doesn't need to globally sample listening demographics in order to disperse a central pool of money according to some arbitrary measure of value. Rather, the money is never pooled -- it goes straight from the fan to the artists (via one of many competing payment gateways). The samples are never taken -- it's not really practical in the first place, and it's just not needed. And no arbitrary measure of value is selected -- it's left up to every fan to decide how much to give his artists.<br> <br> </p> </li> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>What about piracy?<br> </u>What about it? It already happens today in vast amounts, and no plan on the books even claims to have a chance of doing anything about it. Piracy *is* online music -- everything else is just an aberration. This plan seeks to capitalize on the real world as it exists today, tapping into the vast sums of money that fans currently aren't giving to music labels.<br> <br> </p> </li> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>What about privacy?<br> </u>This system gives exceptional privacy protections to all involved because there is no one entity that sees all activity. As such, it doesn't centrally aggregate sampling data, demographic profiling, historical traffic, personally identifiable information, or any of the problems that people are generally skittish about. The centermost entity of this plan is an organization that just has anonymous fingerprints of unnamed songs, and knows absolutely nothing about the songs themselves, the artists who make them, the users who listen to them, or the interactions in between.</p> </li> </ol> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.5in;"><br> </p> <ol start="7"> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>X got paid $Y before, will he still be?</u><br> Possibly. Maybe he'll get paid more. Or maybe less. The same can be said about every other solution on the table.</p> </li> </ol> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.5in;"><br> </p> <ol start="8"> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>But it's not fair! How will X get paid for Y?<br> </u>This plan recognizes that every fan has a different idea of what is or is not fair, and fully empowers him to act upon that notion. Even the old system that is rapidly dying wasn't "fair", it's merely "what was". This plan does not attempt to blindly copy what was, nor invent some new notion of "fair" and mandate that all fans obey it under threat of force. So in this sense, it is arguably the most fair of all.<br> <br> </p> </li> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>Hasn't this been tried before?</u><br> Everything's been tried before, and everything has failed – all plans have failed – due to lack of support and outright opposition by “old guard” music industry. Virtually every innovative plan, both voluntary and compulsory, has been crippled through lawsuit, squeezed through impossible pricing, or bypassed through refusal to participate. There's very little in this plan that's new, and without action by the existing industry, this plan to create a feasible commercial alternative to raw, uncompensated piracy will fail just like all the others have and are failing. But this proposal isn't intended as a panacea. It's intended as a review of what's possible should the music industry decide to begin acting reasonably and in the interests of artists, fans, and society at large. There are signs that the industry is starting to have reason forced upon it by investors, artists, and even a gradual awakening of common sense after a decade of complete destruction of shareholder value. One day, they will either become irrelevant or will sign up to one of the many, many plans proposed and nurtured over the years. Maybe they'll choose this one. Maybe not. The point of you reading this is to be aware that the vision presented herein is in fact possible, and to either encourage the industry to adopt this proposal, or to encourage congress to strip the industry of its abused and overzealous tools of copyright enforcement such that we can continue on without them. How many more decades are we willing to wait?</p> </li> </ol> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.5in;"><br> </p> <ol start="10"> <li> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt;"><u>So that's all well and good, but seriously... Where's the sampling?<br> </u>Seriously, it's not needed. Take it in reverse.</p> </li> </ol> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">Q: Why sample?</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">A: Well, we know how to at least try to sample music fingerprints transferred over the backbone, and we think that samples are somehow related to how often songs are listened to, so by sampling we can get a sense of which songs are most often listened to.</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">Q: Why do we care how often songs are listened to?</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">A: Well, we're assuming that the number of times a song is listened to is representative of how valuable it is to fans.</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">Q: Why do we care if a song is valuable to fans?</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">A: Because artists must be paid in proportion to value, obviously!</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">Q: Paid by whom?</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">A: Well... by fans, I guess... obviously.</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">Q: Why don't fans pay artists directly?</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">A: Well they *were*, through CD sales, until piracy ruined everything.</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">Q: I thought CD sales largely didn't go to artists.</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">A: Well... if you want to get *technical*, no, but they sorta "trickled down to artists"... It's complicated.</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">Q: Ok, again, why don't they pay artists directly?</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">A: Because that's impossible! What, are they supposed to track down every artist in their playlist and give them a nickel each time they play the song?</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">Q: Sure, why not?</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">A: Because... because you just can't. It's complicated. Fans can't be trusted to support their artists directly. They need help.</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">Q: Help from whom?</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">A: Well, help from me, of course. And my friends. Only we can get artists compensated.</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">Q: But I thought your CD sales largely didn't go to artists?</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">A: Yes they do! They trickle!</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">Q: So let me get this straight: the goal is to help artists get paid by fans in proportion to how much fans like them. But fans can't be trusted to do it directly, and instead artists need the help of organizations that historically take the lion's share of the profit and leave a trickle for the artists themselves? And the best way to do this is to force everyone to pay you a bunch of money that you distribute based on relative estimated value to fans calculated by sampling backbone traffic for a small set of music fingerprints, extrapolating global traffic, inferring total music listens from that, and then converting that sampled/extrapolated/inferred number into "value to fans" with an arbitrary formula selected by... by whom again?</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">A: By me.</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">Q: Got it.</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">A: That's right! Now you're getting it.</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">Q: And why not just let fans give artists money directly?</p> <p class="western" style="margin-bottom: 0in; direction: ltr; color: rgb(0, 0, 0); font-family: Helvetica,sans-serif; font-size: 10pt; margin-left: 0.98in;">A: You just... you just can't! And... it's different, and therefore scary. Artists talking to fans? Fans talking to artists? What an absurd thought. Fans can't be trusted! Artists don't want to talk to fans! There need to be a middleman. Lots and lots of middlemen. And formulas! And sampling! And most importantly -- a huge, enormous pool of money. That I control. Trust the trickle. It worked for your grandpa. Why can't it work for you?</p> </span><br> David Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com0tag:blogger.com,1999:blog-8655656.post-31050737314829734142011-01-24T22:31:00.000-08:002011-01-24T22:32:00.745-08:00NAT penetration algorithm from iGlance, circa 2005How's this for a blast from the past. I posted this to the iGlance <br>Yahoo group. You can read the original here, which has a couple <br>follow-up replies:<p><a href="http://tech.groups.yahoo.com/group/iglance/message/52">http://tech.groups.yahoo.com/group/iglance/message/52</a><p><br>But here's the text itself for your reading pleasure:<br>-----------------------------------------------------------------------<br>Hi, thanks for writing. NAT penetration is a very tricky subject, so<br>let me first give an overview of what the obstacles are, and then I'll<br>explain my approach for circumventing them.<p>(Note, the 'STUN' protocol I'm using is home-brewed -- it's not not<br>truly compliant with RFC3489, for reasons I can get into if you care to<br>hear. However, it accomplishes the same thing.)<p>First, assume the following network:<p>+--------+ +-----+ +--------+<br>| Client | ---> | NAT | ---> | Server |<br>+--------+ +-----+ +--------+<p>The client is connected to the NAT, and the NAT is connected (via the<br>internet) to the server. The client is generally on some LAN, and thus<br>has a "private" IP address. However, the NAT is generally on the<br>internet, and thus has a "public" internet IP address. Thus while the<br>client cannot send packets directly to the server (because the client<br>isn't on the internet), the client can send it "through" the NAT.<p>Now, UDP packets indicate from which address they originated. But which<br>address does the packet appear to be from when the server receives it:<br>the client, or the NAT? The answer is the NAT -- NAT stands for<br>"Network Address Translator" because it translates "private" addresses<br>(such as on a LAN) to "public" addresses (such as on the internet).<p>So the client sends a packet from the LAN address (call it privateIP)<br>but the server thinks it's coming from an internet address (call it<br>publicIP) due to the NAT's translation. So long as the client is<br>simply sending to the server, there's no problem -- if the<br>server is only receiving, it doesn't care what address the packet comes<br>from. But the moment the server wants to reply, then things get tricky.<p>In the easy case when a server is replying to a client request, the<br>server just sends back to the address the request packet appeared to<br>come from (ie, the publicIP). And when the NAT receives it, it forwards<br>it back to the client. In this way, when a client establishes a<br>connection with a server, the client and server can talk back and forth<br>without trouble.<p>However, the reverse is not so easy. Now, when the client initiates a<br>connection with the server, it 'punches a hole' through the NAT. This<br>hole (also called a 'mapping') is what the server uses to talk back with<br>the client. However, if the client doesn't punch the hole to the server<br>first, the server can't contact the client. Indeed, if the server sends<br>a packet to 'publicIP' before the client punches the hole through the<br>NAT, the NAT will just silently disregard the message and it'll never<br>arrive.<p>Thus a NAT is a bit like a one-way mirror: a client behind a NAT can<br>contact servers without restriction, but servers can't do the same.<br>Many people like this behavior for security reasons. But obviously, in<br>a P2P network this is less desirable because if you're behind a NAT, a<br>remote client can't contact you until you contact it. But if it's also<br>behind a NAT, you can't contact it until it contacts you. A seemingly<br>intractable problem.<p>To solve this problem, iGlance uses a directory server that acts as an<br>intermediary to help clients behind NATs and firewalls connect directly.<br>The process works as follows:<p>1) Client A connects to the global server and registers its IP<br>2) Client B connects to the global server and asks for the IP for A<br>3) The server informs A that B is trying to contact it<br>4) Client A begins trying to contact B<br>5) Client B begins trying to contact A<br>6) Eventually a direct connection is established<p>As mentioned before, whether A tries to contact B or B tries to contact<br>A, both will fail independently. But when they both try to contact each<br>other simultaneously, they both "punch holes" through their NATs and<br>firewalls, and thus both let the other's communications through. This<br>technique of simultaneous hole punching is the essence of NAT-to-NAT<br>traversal.<p>However, recall that each client typically only knows its "private" IP<br>address -- ie, the IP address on its private LAN. But just as the<br>server sees only a client's "public" IP address, so do peers only see<br>other peers' public IPs. Thus before client A can attempt to contact<br>client B, A needs to learn B's public IP.<p>This process of a client determining whether or not it is behind a NAT<br>(and if so, finding its public IP address) is called the 'STUN' process<br>-- named after the IETF standard RFC3489. (iGlance doesn't use this<br>protocol, but is heavily influenced by it.) The precise technique<br>iGlance uses is as follows:<p>1) STUN server is assigned 3 IP addresses -- STUN0-2<p>2) Client sends STUN request to STUN0<p>3) Client punches hole to STUN1<p>4) The STUN server attempts to contact the client *from* STUN0-2<p>Thus the STUN server sends *three* responses from *three* different<br>IP:port combinations, to the *same* IP:port from which the client<br>request originated. Depending on the NAT and firewall in place, the<br>client might successfully receive up to 3 responses, one each from a<br>different IP:port on the STUN server. Based on which requests succeed,<br>we can guess which type of NAT is between the client and the STUN<br>server. This is used to set the 'Connection_Class' as follows:<p>FIREWALL: (0 responses)<br>Something is blocking either all outbound or inbound UDP traffic.<p>SYMMETRIC: (1 response from STUN0)<br>The client can receive UDP only from the exact IP it sends to.<p>RESTRICTED: (2 responses, from STUN0 and STUN1)<br>The client can receive UDP only from remote IP:ports for which holes<br>have explicitly been punched.<p>UNRESTRICTED: (3 responses)<br>Once a hole is punched through the NAT, any remote IP:port can use it to<br>contact the client.<p>PUBLIC: (3 responses)<br>The client is not behind a NAT and thus can receive from any IP:port.<p>Furthermore, the server returns in the STUN response the apparent<br>IP:port from which the client's request appeared to originate. Recall,<br>the client sends from its 'private' address, while the server receives<br>from the client's 'public' address. If these are different, we know a<br>NAT must be in place. But if they are the same, then we can assume<br>there is no NAT in place and thus the client is connected to the<br>internet directly. (This is how iGlance distinguishes between the<br>UNRESTRICTED and PUBLIC states.)<p>(All this logic is contained in the file GDispatchService.cpp. The STUN<br>request is sent in the function GDispatchService::_requestStun( ), and<br>the responses are processed by GDispatchService::_onInput( ) in the<br>GDSS_STUN state.)<p><br>So clients with PUBLIC, UNRESTRICTED, or RESTRICTED NATs know they can<br>receive UDP directly from another peer. And clients behind SYMMETRIC<br>NATs or UDP-blocking FIREWALL know they can't (they must establish a<br>'TURN' connection with the server, which simply listens for UDP traffic<br>and sends back over HTTP). Armed with this information, clients can<br>ensure they are able to be contacted by remote peers, whether behind a<br>NAT or FIREWALL, or directly on the internet.<p><br>Does this answer your question?<p>-davidDavid Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com0tag:blogger.com,1999:blog-8655656.post-40787514607317140982011-01-22T10:35:00.000-08:002011-01-22T10:36:04.646-08:00Why hasn't anybody built FreePandora yet?Anybody know anything about this? Care to take any guesses?<p><a href="http://torrentfreak.com/the-music-bay-pirate-bay-110122/">http://torrentfreak.com/the-music-bay-pirate-bay-110122/</a><p>TPB has never really been a coding organization, so my bet is it's not <br>on some amazing new P2P service, but rather just a retooled version of <br>TPB website that is optimized for music content. In other words, the <br>basic foundation will still be a standard Torrent client.<p>I'm still amazed nobody has built a really good pirate music outfit -- a <br>*true* Pandora whose box when opened can't ever be closed. Music as a <br>product only has two real features:<p>1) Play this song (or list of songs) right now<br> - Search MusicBrainz<br> - Find the most popular album containing the song<br> - Search TPB for the highest seeded version of that album<br> - Download it with libtorrent<br> - Fish out the song you actually wanted<br> - Play<br>2) Play songs around this theme until I tell you to stop<br> - Given an artist name<br> - Look for similar artists on MusicBrainz<br> - Assemble a big playlist<br> - Download albums one at a time like (1)<br> - Play a random mix of whatever subset is available<br> - Keep expanding that subset<p>It's the simplest possible product to conceive. All the hard problems <br>have already been solved: the content is readily available, the metadata <br>is already there. All the pieces are in place and are just waiting for <br>someone to assemble it into a user-friendly package. The only "work" <br>involved is:<p>1) Build a UI with three input elements:<br> . A search box<br> . A "Play exact" button<br> . A "Play like" button<p>2) When "Play exact" is pressed it goes out, downloads, and plays that <br>exact song, artist, album, etc. Furthermore, if it's already downloaded <br>it just plays from its cache.<p>3) When "Play like" is pressed, it instead goes and finds a range of <br>songs/artists/albums like it, and plays those instead.<p>The only challenge is dealing with mapping the fuzzy input from the user <br>into MusicBrainz, and then mapping its output into ThePirateBay, and <br>then figuring out which song downloaded is the one you want. But again, <br>that's a solved problem. I don't personally know the best solution, but <br>if you convert everything into soundex sequences and just match based on <br>how many common homophones it has, I bet you'd get pretty close.<p>Anybody on this list could build it. Seriously, it is a one-person job, <br>and there are probably dozens of people on the list with the time, <br>energy, and inclination to do it. Why study some esoteric P2P mesh <br>problem that odds are won't ever matter, when in the same (or less) time <br>you could build a world-shaking music service, single-handedly? You <br>could be *the guy* to take down the music industry.<p>Especially if you're in a non-US jurisdiction, this seems a no brainer.<p>Anyway, maybe ThePirateBay will do this now, but I doubt it. I expect <br>we'll need to wait for some nameless individual on the other side of the <br>world to step up. It really, truly, only takes one person to change the <br>world, forever.<p>-davidDavid Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com0tag:blogger.com,1999:blog-8655656.post-54388012553396698272011-01-17T22:59:00.001-08:002011-01-17T22:59:41.900-08:00More mesh thoughtsI think the key question, as has always been the question when it comes <br>to P2P network, is usability. Skype "just worked" so well that it took <br>off like mad. Same for the major pirate networks (though even those are <br>surprisingly unwieldy). A wireless mesh will only take off if it's <br>absolutely dead simple. In fact... I could see it leveraging some of <br>SocialVPN and the P2P social network concepts. Imagine:<p>1) You buy this USB device from WalMart, and plug it in for the first time.<p>2) An app launches, whether you're on Mac, Windows, Linux, iPad, whatever.<p>3) It asks "Welcome to the Mesh! Do you have an account, or do you want <br>to create a new one?" You choose "Create a new one, named Quinthar"; it <br>generates a huge public key.<p>4) It asks "There are 23 nodes in range named Alice, Bob, Cathy, etc. <br>Which are your friends?" You choose "Alice".<p>5) It shows you Alice's public profile, which is available to anyone. <br>It's up to Alice to decide how much to show. It asks "What password <br>would you like to use to friend Alice?" You say "Wonderland"<p>6) On Alice's computer it says "Quinthar would like to be friends, what <br>is the password?" She asks you, then types it in "Wonderland". It says <br>"Great, now you and Alice are friends, and will stay connected so long <br>as you are directly in range, share an intermediate mesh node, or are <br>both connected to the internet." [Eg, it works just like SocialVPN and <br>if it can't directly connect, establishes a NAT-penetrated connection <br>over the internet. After the initial setup, you never need to think <br>about it again.]<p>6) Once connected, you can see Alice's "Friend" profile, which is shown <br>to anybody who is friends Alice. It might have additional information, <br>such as online status, more photos and such, as she chooses. She sees <br>the same for you.<p>7) It says "Now that you're friends with Alice, what do you want to do?" <br> You say "Share these songs, photos, and videos, but not these other <br>ones." [Perhaps by folder.] When she looks at your profile, she sees <br>all these items. She can set offline preferences to optionally sync <br>your data to her computer for access if you get separated. You might <br>have a variety of access levels that you choose to share or not with <br>different people.<p>8) It says "Great, it's shared with Alice. Do you want to share with <br>any of Alice's friends -- including those you don't know?" You directly <br>set how many levels of indirection you'll allow, perhaps just defaulting <br>to 3 (Alice, Alice's friends, Alice's friends friends.)<p>... fast forward until you have many connections, some of which are <br>physically in range, others are connected via a VPN over the internet, <br>others are offline ...<p>9) You have a vast interface to browse the photos, videos, songs, <br>updates, profile information, and basically a lot of stuff about <br>everybody around you. The USB dongle is used to install on a new <br>computer, and connect directly without the internet, but even without <br>the dongle an installed computer can continue to participate in the mesh <br>via the internet.<p>10) If any particular computer gets lost or compromised, you can <br>unfriend them (or remove just that device) immediately. Furthermore, <br>your node is configured to monitor unfriending to automatically <br>"quarantine" any node that has become suspect. (For example, one of my <br>friends lost his iPhone; he'd remove that device from his profile and my <br>devices would stop talking with it, without any involvement from me.)<p>11) And because your USB dongle is owned by you, it can store data such <br>as your private key so you can easily move it between computers -- or <br>even quickly access your mesh using someone else's computer, without <br>leaving any trace on the computer itself.<p><br>Anyway, ultimately I think mesh technology will be far less important <br>than mesh *usability*. It needs to be packaged up with really simple, <br>excellent software that enables the most basic peer activities -- <br>especially file transfer -- to be done in a totally seamless way<p>-davidDavid Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com0tag:blogger.com,1999:blog-8655656.post-69515010838673391962011-01-16T18:19:00.001-08:002011-01-16T18:19:43.230-08:00Strains within China's leadershipThis is the sort of thing that makes me wonder if China can maintain its <br>current path:<p> <a href="http://www.nytimes.com/2011/01/17/world/asia/17china.html">http://www.nytimes.com/2011/01/17/world/asia/17china.html</a><p>Basically: does authoritarianism scale, or inevitably succumb to <br>internal power struggles? This article would suggest some of the <br>latter, that the Chinese leadership is losing control as the military, <br>finance, and industrial sectors act with increasing autonomy -- <br>sometimes in defiance of the central leaders' will.<p>It's interesting how the conversations about the Chinese rise to power <br>rarely discuss the possibility of true internal dissent, or desperate <br>actions to contain it. That's actually what frightens me most: there's <br>no legitimate reason for the Chinese to go to war with the US or its <br>neighbors, but war is a fantastic distraction from internal dissent.<p>Ultimately I continue to bet on the US because out of all nations, I <br>think we're the best of managing internal content, which is ultimately <br>the greatest threat to any power -- imperial or otherwise.<p>-davidDavid Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com1tag:blogger.com,1999:blog-8655656.post-27510306104074804852011-01-11T23:45:00.001-08:002011-01-11T23:45:09.370-08:00How Piracy Will Hyperlocalize with Mesh NetworksWhile I don't think a pirate mesh is on the near horizon, I do think <br>it's entirely feasible -- and the easiest way to accelerate its arrival <br>will be to inconvenience piracy on the internet. Regardless, as a fun <br>thought exercise I imagine it'll happen like this:<p>1) Somebody packages up a software radio onto a convenient USB stick:<br> <a href="http://en.wikipedia.org/wiki/Software-defined_radio">http://en.wikipedia.org/wiki/Software-defined_radio</a><p>2) Because it's just software, the same hardware can be used for <br>essentially any wireless activity. First it'll probably just be a <br>universal wireless broadband card, with modules to connect to every <br>cellular network in the world. It'll start out targeted to travelers. <br>It'll also include GPS, AM/FM, wifi, and pretty much everything you <br>could want because, well, why not. It's software; it can do it all.<p>3) It'll be instantly "unlocked" by the open source community, assuming <br>it's not in fact built by said community in the first place:<br> <a href="http://gnuradio.org/redmine/wiki/gnuradio">http://gnuradio.org/redmine/wiki/gnuradio</a><p>4) A whole new generation of wireless protocol research will be <br>unleashed by universities and individuals alike, with a clear focus on <br>mesh technology merely because that's the new hotness.<p>5) To start, mesh software will just be run by a few crazy hackers as a <br>background process while using their universal wireless cards, which <br>they use because it's easier than dealing with wifi. Node density will <br>be low and largely limited to toy apps like chat, single file transfer, <br>etc. More in the vein of being a proof of concept.<p>6) There will be some place where a critical mass of node density <br>occurs: probably a university with a combination of a strong engineering <br>school and overzealous network administrator. It'll always be possible <br>for one person to get a torrent off the real internet, but then the rest <br>of the dorm will get it via the mesh.<p>7) The next semester, students who don't really have any idea about the <br>mesh or have any interest in a universal wireless device will realize <br>"if I just buy this thing and let some dude install software on my <br>laptop, I can get a ton of great content without risk of detection by my <br>university." The device's main purpose will gradually transition away <br>from its advertised and intended purpose, and repurposed by pirates.<p>8) This will slowly, quietly grow. The hardware manufacturers will <br>initially be totally unaware, but gradually adopt a policy of "don't ask <br>don't tell". More and more students will sign up. As people go home, <br>students who got their content from the device won't even know how to <br>share it without the device; they'll convince their friends to buy it <br>just so they can easily share the content while home on break.<p>9) The software will get better and better. Torrent apps will <br>auto-detect if the device is there, and will try to pull from it first. <br> The torrent protocol itself will adjust to pull from nearby mesh <br>neighbors. Gradually, piracy will go hyperlocal.<p>10) The hardware will get better and better. All laptops will come with <br>this built in, because why have a dedicated wifi card (or Sprint card) <br>when you can have a single universal card? Why have a card at all when <br>it can be done as part of the main chip? After all, it's just software <br>-- CPUs run software too.<p>11) At some point Apple really starts to take notice. Apple products <br>will recognize a "neighborhood" network that operates across the mesh -- <br>like Bonjour on steroids. It advertises its security and speed <br>advantages over "the internet", which gradually becomes used exclusively <br>for what it's good at -- moving data over incredible distances under <br>watchful eye of the state -- versus the mesh, which is for small <br>distances with anonymity.<p><br>Something like the above *will* happen. It's inevitable. It's not even <br>that creative. And it'll probably happen sooner than we expect. Sound <br>unlikely? Remember those researchers that cracked GSM at the CCC 2 <br>weeks ago? They did it with "Universal Software Radio Peripheral"<p><a href="http://www.zdnet.co.uk/blogs/security-bullet-in-10000166/gsm-crack-inexpensive-says-researcher-10021405/">http://www.zdnet.co.uk/blogs/security-bullet-in-10000166/gsm-crack-inexpensive-says-researcher-10021405/</a><p>You can buy one at <a href="http://www.ettus.com/">http://www.ettus.com/</a> And yes, it plugs in via USB.<p>-davidDavid Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com0tag:blogger.com,1999:blog-8655656.post-75427146160197021052011-01-03T16:08:00.001-08:002011-01-03T16:08:10.166-08:00100M monthly users of BitTorrent/uTorrentSaw this today:<p><a href="http://www.dmwmedia.com/news/2011/01/03/bittorrent-filesharing-apps-hit-100-million-monthly-users">http://www.dmwmedia.com/news/2011/01/03/bittorrent-filesharing-apps-hit-100-million-monthly-users</a><p>That's 100M people who fire up BitTorrent/uTorrent to download <br>*something*. And that doesn't include all the other torrent <br>applications out there. Or IRC/Newsgroup piracy. Or sneakernets.<p>For comparison, Hulu has about 30M monthly users, and Netflix is up to <br>16.9M users last quarter (not sure about monthly):<p><a href="http://mashable.com/2010/11/10/hulu-stats/">http://mashable.com/2010/11/10/hulu-stats/</a><br><a href="http://www.investmentu.com/2010/December/netflix-creates-multibillion-dollar-industry.html">http://www.investmentu.com/2010/December/netflix-creates-multibillion-dollar-industry.html</a><p>Also interesting to check out the growth: Vuze.com up 81% over the last <br>year. uTorrent up 106%. Bittorrent.com up 144%. As opposed to <br><a href="http://hulu.com">hulu.com</a> up 61%. Netflix up 8.23%.<p><a href="http://siteanalytics.compete.com/bittorrent.com+utorrent.com+vuze.com+azureus.sourceforge.net/">http://siteanalytics.compete.com/bittorrent.com+utorrent.com+vuze.com+azureus.sourceforge.net/</a><p>So, piracy still bigger and still growing faster. As has been pretty <br>much consistently the case since the dawn of the internet. I'd wager <br>the trend will hold for another decade or so, if not forever.<p>-davidDavid Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com1tag:blogger.com,1999:blog-8655656.post-23571038833519337012011-01-02T12:26:00.001-08:002011-01-02T12:32:59.890-08:00The increasing price of progress (and how to get a discount)Was reading this fascinating article here:<p><a href="http://www.newyorker.com/reporting/2010/12/13/101213fa_fact_lehrer">http://www.newyorker.com/reporting/2010/12/13/101213fa_fact_lehrer</a><p>And it occurred to me that science and math are typically viewed as <br />having the same goals -- to prove theories about the world around us. <br />But perhaps they should be viewed as opposites? After all, there's no <br />real way to prove *anything* using the scientific method. But you *can* <br />disprove something. Why isn't that the focus? Perhaps math should be <br />about proof, and science should be about *disproof*?<p>Due to the fact that proof (rather than disproof) is the focus of most <br />scientific research today, we end up with a ton of research that rides <br />the thin and ambiguous line between statistical relevance and <br />irrelevance. Indeed, the above article suggests that most scientific <br />"conclusions" are irreproducible nonsense. Literal nonsense. That was <br />created at enormous cost to society.<p>Now you might say "well that's just the price of progress". And that's <br />probably true. But we should be focused on driving that price *down*, <br />when in fact it seems to me that we're driving the price of progress up <br />through irresponsible public policy.<p>There are a lot of reasons why this could be the case. Probably the <br />most direct contributor is the largely well-intentioned but ineffectual <br />policy of promoting amazingly expensive formal education to people who <br />don't want or need it. This fills our research labs and journals with <br />nonsense (nonscience?) studies done in the pointless pursuit of <br />meaningless, debt-inducing degrees. But I think a more damaging and <br />insidious reason is, yes, intellectual property.<p>I think the reason I'm so opposed to copyright and patent** is that <br />those policies actually damage the world. Meaning, they irresponsibly <br />encourage "quantity" over "quality", creating more options of lower <br />quality when fewer high-quality options would have been <br />faster/cheaper/better.<p>** Trademark has a completely different aim: helping consumers correctly <br />differentiate between similar alternatives. Trademark is primarily <br />aimed at increasing "quality".<p>Here you might say "but who will innovate without IP protections?" And <br />I guess I'd say "those who need to". They say "necessity is the mother <br />of invention", not patent protection. In fact, I wonder if IP has done <br />anything *at all* to improve the quality of innovation (or, rather, the <br />quantity of high-quality innovation) on a "per-capita" basis.<p>Sure, we have more innovation today than any any point in human history. <br /> But we also have more *people*. Furthermore, the rate of new people <br />coming into the world is higher than most points in in history. Even if <br />innovation-per-person is constant, today will be more innovative than in <br />the past, in aggregate. So even if IP is a total failure and does <br />absolutely nothing of value, today will still seem very innovative (and <br />those policies still seem a success).<p>But if there were no patents, does anybody honestly think anything we <br />have around us wouldn't exist? Would we have not bothered with steam <br />power, railroads, electricity, phones, cars, rockets, satellites, or any <br />of that? Would we have never noticed any of the major medical <br />breakthroughs? I doubt it. I think we'd have pretty much everything we <br />do now. We'd have them because we *need* them to compete between <br />nations -- in an arena where IP protections don't really exist.<p>Accordingly, I see no evidence whatseover that IP works. I don't know <br />of any major series of breakthroughs that simply wouldn't have happened <br />in roughly the same order at any slower pace without IP. At best it seems <br />just a big nuisance. But my real fear is it's more than just a <br />nuisance. Rather, it's an active damping function on human innovation.<p>I fear the primary effect of patent today is to introduce arbitrary <br />"waiting periods" before old inventions can be compounded into new ones. <br /> It introduces enormous fear, uncertainty, and doubt into the <br />inventor's mind -- a sense of "why should I even bother doing this thing <br />that would be awesome when I'll probably just be sued into personal <br />bankruptcy out of the blue by some nameless corporation?" It's not <br />focused on quality *or* quantity, but creating an unnecessary tollbooth <br />on innovation and then charging society by the mile, with the proceeds <br />not even going to the innovators responsible.<p>Similarly, I fear the whole design of copyright is maliciously misguided <br />on creating "the next big thing" rather than maximizing the <br />accessibility and influence of the untold millions of "past big things". <br /> It holds the output of all past artists hostage -- most of who are <br />dead or who are lucky to make a single thing of widespread appeal in <br />their entire lives -- disingenuously invoking the plight of nameless <br />future artists to justify another unnecessary tollboth, the vast <br />majority of whose proceeds don't go to artists.<p>This isn't a call for communism -- IP shouldn't be shared out of some <br />moral responsibility. And it's not a call for socialism; the government <br />needn't seize private invention for the public gain. It's saying IP is <br />a *detriment* to competition, the most important foundation of <br />capitalism. It's saying private inventors (and the businesses who <br />employ them, and the investors who fund them) would all be better off <br />without IP.<p>The world doesn't need IP. Innovators and artists don't need IP. It <br />was created by those who don't innovate, to control, contain, and profit <br />from those who do. It's just a raw deal for the world. And it needs to <br />be stopped.David Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com1tag:blogger.com,1999:blog-8655656.post-9568740817998834152010-11-24T10:53:00.001-08:002010-11-24T10:53:52.451-08:00Even the WIPO agrees piracy is obvious and unstoppableNot that such a thing would ever reflect itself in our IP policy. But <br>here are some choice quotes:<p><a href="http://www.wipo.int/edocs/mdocs/enforcement/en/wipo_ace_6/wipo_ace_6_5.pdf">http://www.wipo.int/edocs/mdocs/enforcement/en/wipo_ace_6/wipo_ace_6_5.pdf</a><br>------------------------------------------------------------------<br>13. To be more explicit about these limitations, we have seen no <br>evidence — and indeed no claims — that enforcement efforts to date have <br>had any impact on the overall supply of pirated goods. ...<p>14. The debate is also notable for its lack of discussion of the <br>endgame: of how expanded enforcement, whether Internet-based in the form <br>of proposed "three-strikes" laws, or street-based in the form of <br>stronger criminal sanctions, will significantly change this underlying <br>dynamic. ...<p>15. Perhaps most importantly, we see little connection between these <br>efforts and the larger problem of how to foster rich, accessible, legal <br>cultural markets in developing countries — the problem that motivates <br>much of our work. ...<br>------------------------------------------------------------------<p>Alas, just 16 more pages heaped on the thousands already written <br>detailing in exquisite clarity that piracy is not only a natural and <br>reasonable reaction to the absurdity foisted upon the world's content <br>consumers, but no attempts to stop it have been remotely effective, and <br>no current attempts under consideration show any sign of being more <br>effective than their predecessors.<p>-davidDavid Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com0tag:blogger.com,1999:blog-8655656.post-34953721360511608172010-05-28T19:48:00.000-07:002010-05-28T19:50:31.383-07:00Fighting and Winning the Pyrrhic WarThis is my response to:<br /><br /><a href="http://gonze.com/blog/2010/05/28/the-sue-em-all-mystery-solved/">http://gonze.com/blog/2010/05/28/the-sue-em-all-mystery-solved/</a><br /><br />Are you sure it's working out well? All the troubles of TPB amount to little more than a couple slight dips -- from enormous piracy to slightly less enormous piracy. TPB is still alive and well, even if its founders are in hot water:<br /><br /><a href="http://thepiratebay.org">http://thepiratebay.org</a><br /><a href="http://siteanalytics.compete.com/thepiratebay.org">http://siteanalytics.compete.com/thepiratebay.org</a><br /><br />isohunt.com made the mistake of operating from the US, but it's not like shutting it down will do anything other than cause those users to go elsewhere. And it's worth noting it hasn't been shut down either:<br /><br /><a href="http://isohunt.com/lite/#q=avatar">http://isohunt.com/lite/#q=avatar</a><br /><a href="http://siteanalytics.compete.com/isohunt.com/">http://siteanalytics.compete.com/isohunt.com/</a><br /><br />As for the next rung, Demonoid seems to be gaining steam -- perhaps picking up isohunt and TPB users who are hopping ship:<br /><br /><a href="http://siteanalytics.compete.com/demonoid.com">http://siteanalytics.compete.com/demonoid.com</a><br /><br />And the third rung is looking healthy too. Mininova took a hit, but their both seeing decent growth otherwise:<br /><br /><a href="http://siteanalytics.compete.com/btjunkie.com+mininova.com/">http://siteanalytics.compete.com/btjunkie.com+mininova.com/</a><br /><br />In short, nothing has been remotely effective at reducing torrent pirating. Indeed, the only tangible effect is now there are actual *pirate political parties* with seats in major world governments. Does that seem like a force on the decline to you?<br />As for Limewire, the only thing surprising by it is that it took a *decade* for such an overtly infringing and inducing product to come under threat. The law is still barely able to fight late-90's technology. Speaking of which, it's worth noting that you can still download and use Limewire today, not to mention all the obvious GPL clones that both already exist and will continue to exist:<br /><br /><a href="http://www.limewire.com/">http://www.limewire.com/</a><br /><br />As for that Viacom case against the world's largest pirate, YouTube in its early days, how's that going?<br /><br /><a href="http://www.tomsguide.com/us/Viacom-Google-P2P-Downloads,news-6943.html">http://www.tomsguide.com/us/Viacom-Google-P2P-Downloads,news-6943.html</a><br /><br />The only party "winning" against pirates are businesses who hemorrhage money giving content away at a loss (or with business models nobody feels are remotely sustainable).<br /><br />Indeed, in what possible universe can you claim pirates aren't demolishing their foes in every field of battle they choose to fight? (Streaming being the notable example of a field they haven't yet taken an interest in.)<br /><br />The only battles the copyright forces are winning are Pyrrhic. They're *masters* of those.<br /><br />-davidDavid Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com0tag:blogger.com,1999:blog-8655656.post-26729017268171749642010-03-30T18:49:00.001-07:002010-11-24T10:55:22.631-08:00Nobody cares about climate change...... because of stupid things like this. Seriously? Anybody who calls <br />for a suspension of the worlds' democracies in order to fight climate <br />change is an idiot. Don't get me wrong -- it'd take that (and more) to <br />actually do anything about it. But the rational response to that <br />scenario isn't to call for the impossible (and thus brand yourself <br />irrational), but to say "There's probably nothing humanity can do to <br />stave off climate change, so let's just plan on it occurring and prepare."<p><a href="http://www.guardian.co.uk/science/2010/mar/29/james-lovelock-climate-change">http://www.guardian.co.uk/science/2010/mar/29/james-lovelock-climate-change</a><p>Humans are too stupid to prevent climate change from radically impacting <br />on our lives over the coming decades. This is the stark conclusion of <br />James Lovelock, the globally respected environmental thinker and <br />independent scientist who developed the Gaia theory.<p>It follows a tumultuous few months in which public opinion on efforts to <br />tackle climate change has been undermined by events such as the climate <br />scientists' emails leaked from the University of East Anglia (UEA) and <br />the failure of the Copenhagen climate summit.<p>"I don't think we're yet evolved to the point where we're clever enough <br />to handle a complex a situation as climate change," said Lovelock in his <br />first in-depth interview since the theft of the UEA emails last <br />November. "The inertia of humans is so huge that you can't really do <br />anything meaningful."<p>One of the main obstructions to meaningful action is "modern democracy", <br />he added. "Even the best democracies agree that when a major war <br />approaches, democracy must be put on hold for the time being. I have a <br />feeling that climate change may be an issue as severe as a war. It may <br />be necessary to put democracy on hold for a while."<p>Lovelock, 90, believes the world's best hope is to invest in adaptation <br />measures, such as building sea defences around the cities that are most <br />vulnerable to sea-level rises. He thinks only a catastrophic event would <br />now persuade humanity to take the threat of climate change seriously <br />enough, such as the collapse of a giant glacier in Antarctica, such as <br />the Pine Island glacier, which would immediately push up sea level.<p>"That would be the sort of event that would change public opinion," he <br />said. "Or a return of the dust bowl in the mid-west. Another <br />Intergovernmental Panel on Climate Change (IPCC) report won't be enough. <br />We'll just argue over it like now." The IPCC's 2007 report concluded <br />that there was a 90% chance that greenhouse gas emissions from human <br />activities are causing global warming, but the panel has been criticised <br />over a mistaken claim that all Himalayan glaciers could melt by 2030.<p>Lovelock says the events of the recent months have seen him warming to <br />the efforts of the "good" climate sceptics: "What I like about sceptics <br />is that in good science you need critics that make you think: 'Crumbs, <br />have I made a mistake here?' If you don't have that continuously, you <br />really are up the creek. The good sceptics have done a good service, but <br />some of the mad ones I think have not done anyone any favours. You need <br />sceptics, especially when the science gets very big and monolithic."<p>Lovelock, who 40 years ago originated the idea that the planet is a <br />giant, self-regulating organism – the so-called Gaia theory – added that <br />he has little sympathy for the climate scientists caught up in the UEA <br />email scandal. He said he had not read the original emails – "I felt <br />reluctant to pry" – but that their reported content had left him feeling <br />"utterly disgusted".<p>"Fudging the data in any way whatsoever is quite literally a sin against <br />the holy ghost of science," he said. "I'm not religious, but I put it <br />that way because I feel so strongly. It's the one thing you do not ever <br />do. You've got to have standards."David Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com0tag:blogger.com,1999:blog-8655656.post-14327822789695056132010-02-26T21:30:00.001-08:002010-02-27T04:56:05.272-08:00What is time? A slice of a 4D universe.Just read this article in Wired about time:<p> <a href="http://www.wired.com/wiredscience/2010/02/what-is-time/">http://www.wired.com/wiredscience/2010/02/what-is-time/</a><p>Like every geek, it's a concept that's fascinated me. And like<br />everyone, I have no real clue. But here's my theory nonetheless:<p> Time is just a slice of a 4-dimensional universe.<p>Said another way, the universe is a four-dimensional, static block, and<br />any particular "point in time" is just a slice through the middle. Make<br />sense? No? Let me try to build it up.<p>Imagine a one-dimensional universe. There's up and down, but nothing<br />else. Just one dimension. If you're a dot in that universe, you can<br />move up, then back down, and that's it. However, the very notion of<br />"motion" implies time -- at some time in the "future" you're in a<br />different position than a different point in the "past". So just like<br />up and down, time is also a dimension, with two directions: future and<br />past. So as a dot in a one-dimensional universe, you're actually moving<br />in two directions: up/down, and past/future.<p>Accordingly, this one-dimensional universe, if it can change, is<br />actually more accurately described as a two-dimensional universe.<br />There's up/down, and past/future. Two directions, and you're moving in<br />both.<p>This universe is easy to visualize: think of a heart monitor. Any<br />individual number (eg, heart voltage) is a one-dimensional universe. At<br />any point in time, it has a single value. But that value can change<br />over time. As such, you can easily plot it on paper up/down corresponds<br />to the heart voltage, left/right corresponds to time. Your<br />one-dimensional universe + time has been perfectly captured as a single<br />two-dimensional piece of paper.<p>So we've got this 2D representation of a 1D universe + time. But why<br />are we treating time as a special dimension? Why not just say we have a<br />2D printout of a 2D universe -- one dimension being up/down, the other<br />dimension being time. The 2D universe is the *whole thing*. The only<br />reason we ever saw it as a 1D universe was because we were explicitly<br />ignoring the time dimension.<p>Indeed, the only way to see a heart monitor as anything other than a 2D<br />universe is to explicitly focus on one slice of the page and ignore the<br />others. We call that slice "time". And the fact that different slices<br />of the page map to different "times" is merely a matter of observation<br />-- the paper itself doesn't change. The 2D universe itself is totally<br />stationary, fixed, static, and unchanging. Only our point of<br />observation changes; the only change is *us* changing what part we're<br />looking at.<p>For example, imagine you took another piece of paper, with a tall slice<br />cut through it, and laid it over the first. It would show a single<br />slice of time on that paper; it'd show the position of the dot at one<br />"point in time". Slide the paper from left to right and the point<br />appears to move -- even though in fact the paper underneath it isn't<br />moving at all; only the viewpoint is moving.<p>So from this perspective, the change of time isn't an attribute of the<br />universe. It's an attribute of the *viewer*. The universe -- in this<br />case, a 2D printout of a heart monitor -- is totally unchanging. Only<br />our view of it changes.<p>Now let's add another dimension: let's do a stack of sheets of paper,<br />like a book. Each page is a 2D "slice" of a 3D universe (2D + time).<br />Each page has dots arranged in a particular way, and any page can have a<br />completely different arrangement of dots. To see how those dots "move"<br />we just flip our thumb through the book. Each dot appears to "move" up<br />and down, left and right, when in fact it's not the dots moving -- its<br />our thumb moving, showing us one page at a time. It's our *attention*<br />moving, *experiencing* one page at a time. The first and last page will<br />never change; but our *experience* changes over "time". Once again, the<br />change of time is not an attribute of the book; it's an attribute of *us*.<p>And this has a natural correlation with 4D. Every moment of our current<br />universe is like a page in a big 4D book. This very moment is page 100;<br />a second ago was page 99, and a second from now will be 101. Those<br />pages are totally static -- pages 1, 20, 99, and 100 are written. But<br />so are pages 101, 110, 10000, and 1000000. Our 4D universe (3D + time)<br />is totally static. The only reason it seems to change is because we're<br />only looking at one page "at a time".<p>At least, that's what I think.<p><br />So there are a few FAQ corollaries that come out of this:<p>- Is the universe deterministic? Yes. Every page in the future is a<br />direct consequence of the pages in the past. If you were to somehow<br />step outside the universe and look at the "page" corresponding to this<br />moment in time, you could completely and wholly predict the next or any<br />future page. I also expect you could predict every previous page.<br />Basically, if you were smart enough, with complete knowledge of the<br />current state (page) of the universe you should be able to predict any<br />past or future state.<p>- What about free will? Doesn't exist. Every action you will ever do<br />is pre-ordained and dictated by physics. You and everything you will<br />ever do is purely the consequence of actions that have come before you.<br /> All those things you think you can take credit for? Sorry. Total<br />chance. But hey, all those things that went wrong, they're not your<br />fault either. We're all in it together, everybody a product of the past.<p>- Wait, seriously? Yes, seriously. Free will doesn't "exist" in the<br />sense that you can make some decision that isn't pre-determined by<br />physics. We're just characters in a book that's already been written.<br />But don't feel bad: though the book is written, we're all reading it<br />together. I have no idea what comes next chapter, and nor do you. So<br />it's still exciting to be alive! Free will ultimately doesn't matter<br />(to me, at least). It still *feels* like I'm deciding, discovering,<br />living, and experiencing. So why fret about the metaphysical details?<p>- If time is a dimension, why can't we look that way? Great question!<br />I've always wondered that. My best guess is because all those things we<br />perceive as dots -- molecules, atoms, sub-atomic particles, etc -- are<br />actually lines. And all those lines run mostly parallel, in the<br />direction of time. So our perception of time is to view things<br />perpendicular to time, because time is actually the least interesting of<br />the 4 dimensions. I mean, consider the room you're in now -- the vast<br />majority of it isn't "moving". If you imagine every particle is<br />actually some long wire -- with one end in the far past, one in the far<br />future, and you just seeing a tiny slice of it -- that wire is totally<br />straight. It's super boring. Even those things that are moving are<br />moving pretty slow. Imagine you actually *could* look forward along<br />time -- all you'd see are a series of nearly parallel wires extending<br />off into the future. It's not nearly as interesting to look that<br />direction as to look the other directions. Accordingly, I think we look<br />in the other 3 dimensions because time is boring (and there's no<br />evolutionary advantage to looking forward).<p>- What the hell are you talking about? It's hard to know. It's more of<br />a visual exercise -- viewing the universe as a static, unchanging<br />four-dimensional block, and as us just being some razor-thin slice<br />moving through that block along the time axis. (But not really "moving"<br />-- the part of me that existed a minute ago is still there, one minute<br />behind "me" right now. And all my future me's are up there waiting for<br />me, patiently. Consciousness being like some electric current running<br />along these time-aligned wires, interacting with the other currents<br />running along the wires nearby.)<p>- Ok, so this wire theory is crazy. Ya, but it creates some interesting<br />sub-theories. Like, isn't it strange how the perception of time changes<br />the faster you move? And how the perception of time in theory stops<br />when you're moving the speed of light? Maybe when you're moving the<br />"speed of light" in the three-dimensional space, *there's no more wire*<br />to move in the 4th dimension. You're essentially moving perpendicular<br />to the fourth dimension. Stick with me: the local universe around you<br />is like a bundle of wires, all woven tightly together. If you move<br />slowly together, your wire gradually weaves its way through the<br />super-bundle of the entire universe, eventually making it over to some<br />distant position. But to go faster, you need to bend your bundle at a<br />greater angle. To move super-fast, you need to actually bend your wires<br />at a 90 degree angle -- meaning from everyone else's perspective, your<br />"wires" no longer move at all in the time dimension; they're *only*<br />moving in the other 3 dimensions. To them, you've disappeared. But<br />within your bundle, everything seems fine. The relative arrangement of <br />wires within your bundle seems normal -- all the wires keep going<br />somewhere, and your conscionsness is traveling along those lines at some<br />constant speed. But your wires are no longer aligned with the time<br />axis, so your "local time" seems normal even though it's totally out of<br />whack with the "global time". Which means global time itself isn't<br />really a constant -- it's just the direction that all the other wires<br />typically go, unless they're moving super fast in the other 3<br />dimensions.<p>- So is time a position relative to space, or relative to the wire? Ok, <br />my terminology is getting bad here: the "time" of any particle isn't <br />"absolute distance from the start of the universe", it's "distance from <br />the start of the universe *along that wire*". Imagine two wires, both <br />starting in the same place (the start of time), both perfectly straight <br />and parallel. Their "times" are aligned in that 1 hour in the future, <br />an equal amount of their "wire" has unrolled. But if one "moves" <br />relative to the other, it just means that the wire bends away from the <br />other in 3d space. The further it bends, the "faster" it's moving in 3d <br />space. And that speed comes *from* the time dimension. The fastest you <br />can possibly move in 3D space is to go perpendicular to time. So in <br />theory, if one atom/wire were to turn 90 degrees and run perpendicular <br />to the time dimension for a while, and then turn around and come back to <br />to its original position in 3d space, it could resume its previous <br />arrangement -- except one "hour" of wire would have unfurled for the <br />first atom even though a ton more might have unfurled for the other.<p>- But what this really means is that the "time" dimension isn't actually <br />a special one in any way. It just happens to be the direction that most <br />of the universe's wires are aligned. Had they aligned in a different <br />direction, that would be the "time" dimension. But if a bundle of those <br />wires breaks off in a different direction, it "accelerates" in the 3 <br />other directions while "decelerating" in the time direction. Within <br />that bundle everything seems totally normal -- even though relative to <br />the other bundles it seems "wow, it's moving *really* fast in 3 <br />dimensions and *really slow* in the fourth". I haven't really worked <br />out the math, but I wonder if this is at all consistent with relativity <br />theory.<p>- Isn't this called string theory? I have no idea -- I don't know<br />anything about string theory so I can't say. I'm using "wires" as the<br />metaphor to differentiate my theory from that, until I'm shown they're<br />the same. But I think string theory is about strange vibrations. My<br />wires don't wiggle.<p>- But doesn't quantum theory say true randomness exists? I don't think<br />so. All I know is Einstein said "God doesn't roll dice." Yes, he was<br />an atheist (as am I), but I take it to mean he didn't believe in<br />subatomic randomness either. There have been a lot of things people<br />assumed were random, until we just figured out they weren't. I think<br />it's time to start assuming the opposite. Especially when most<br />pop-science theories of quantum randomness are really just scraping for<br />any possible way to justify an irrational, pseudo-scientific belief in<br />God, free will, self determination, etc.<p><br />Anyway. Gotta run. One thing I've determined is good wine doesn't<br />drink itself. Thank God.<p>-davidDavid Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com0tag:blogger.com,1999:blog-8655656.post-33308319556454386052010-01-24T16:52:00.001-08:002010-02-27T04:55:47.009-08:00IFPI says 95% of downloads illegalI was discussing on a private mailing list whether or not this number is <br />for real. Someone complained it wasn't. Here's my response:<p>-------------<p>Your utter certainty regarding the flawed methodology of their results <br />is somewhat ironic given that you've done no apparent research into how <br />they came to those results, nor any suggestion of what a more accurate <br />number might be.<p>A real critique would go to the source of their data and show why it's <br />bad. I'll help you with that. I found this year's report by simply <br />updating the year in the link from last year:<p><a href="http://www.ifpi.org/content/section_resources/dmr2010.html">http://www.ifpi.org/content/section_resources/dmr2010.html</a><p>The actual quote from there is:<p>> Estimates on the impact of internet piracy vary but are consistently huge in scale. IFPI, collating separate studies in 16 countries over a four-year period, estimated unauthorised file-sharing at over 40 billion files in 2008. This means that globally around 95 per cent of music tracks are downloaded without payment to the artist or the music company that produced them.<p>When read in the actual context, they seem to admit it's a hard thing, <br />and acknowledge they're estimating. Furthermore, contrary to your claim <br />that it was pulled out of thin air, they kindly cite a variety of other <br />sources. I haven't dug into them all, but one is:<p><a href="http://www.ipoque.com/resources/internet-studies/internet-study-2008_2009">http://www.ipoque.com/resources/internet-studies/internet-study-2008_2009</a><p>The paper is 14 pages long with lots of interesting data, as well as a <br />completely description of its methodology. A weakness of their data is <br />they didn't cover North America or Western Europe as a whole (maybe due <br />to data privacy laws?). But focusing on Germany alone (which seems <br />middle of the road in terms of its data compared to other regions), it <br />says 53% of traffic was P2P downloads (separately from VoIP), to 26% web.<p>So right off the bat, P2P downloads account for double the traffic of <br />*all of HTTP* across their not insignificant sample set.<p>It doesn't seem unreasonable to assume that, say, 90% of that P2P <br />traffic was pirate. (It's what it was designed to do, after all.) So <br />let's say 47% of all traffic was pirate content downloads.<p>As for legitimate content downloads, we could probably back into that by <br />estimating bandwidth consumed by iTunes using public sales numbers and <br />average file sizes. But let's say iTunes accounts for 10% of *all HTTP* <br />(which seems astronomically high). That would mean 2.6% of the internet <br />is legitimate content downloads (separate from streaming).<p>47 + 2.6 = 49.6% of the internet devoted to content downloads<br />47/49.6 = 94.75% content downloads pirated.<p>Huh, I didn't even plan that out.<p>Anyway, there are obviously problems with all that data, how it's <br />collected, how it's analyzed, etc. But to categorically assert that the <br />data and results are flawed and everyone involved in the process is <br />either actively lying or allowing themselves to be mislead -- without <br />providing any evidence or analysis to the contrary... let's just say <br />your own methodology could use a bit more scientific rigor.<p>-davidDavid Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com0tag:blogger.com,1999:blog-8655656.post-25070737608450897462009-12-20T16:59:00.001-08:002009-12-20T17:03:23.173-08:00Screw PHP, C++ FTW<a href="http://www.webtoolkit.eu/wt">http://www.webtoolkit.eu/wt</a><p>Basically it's a graphical UI framework (like Qt), except rather than <br />outputting to a windowing interface, it outputs to the web. It's a <br />pretty crazy shift in how you program for the web, but possibly a shift <br />for the better.<p>Basically, PHP and most web frameworks were built around the "page" <br />metaphor, reflecting their content-centric history. Websites sorta back <br />into applications by breaking them down into "pages". But this doesn't <br />always make sense. (In Expensify's case, for example, we pretty much <br />only have two pages that do everything, which is a bit of a mess.)<p>Wt takes an entirely different approach and builds up the application in <br />terms of normal UI constructs. Accordingly, you don't think in terms of <br />page loads and ajax calls, you think in terms of dialogs and frames.<p>Furthermore, it handles all the details of progressively degrading based <br />on browser capabilities. For example, it includes vector rendering. If <br />the browser supports it natively, it'll do it all natively. Otherwise <br />it does it serverside. The programmer can ignore those low-level <br />browser-specific details and just focus on "I want a graphic here that <br />does this".<p>Anyway, probably not a real option right now, but an interesting thing <br />to consider nonetheless. So much of the web is based on its <br />content-centric background, and that makes a lot of it annoying. It's <br />interesting to instead rethink the web not as a series of linked pages, <br />but as a general rendering framework -- like OpenGL or X Windows.<p>Unfortunately, I bet Wt will be overlooked because it's written in C++. <br /> But I think the ideas it's pursuing (though I doubt they pioneered <br />them) are going to become mainstream.<p>Indeed, given the realities of compile-before-deploy (GWT, less css, <br />minification, code generation), the growing adoption of strict typing <br />(eg, in JavaScript 2.0 / ECMScript 4.0), the latest trends in graceful <br />degredation for different form factors and browser capabilities, and <br />even the overwhelming success of Objective-C for iPhone apps -- maybe <br />we're coming to realize that all those tried-and-true language features <br />and programming constructs weren't such a bad idea after all?<p>-davidDavid Barretthttp://www.blogger.com/profile/06665251639022075770noreply@blogger.com0