[Corpora-List] Copyright question again

Tue Jan 6 06:15:29 UTC 2015

Word lists and frequency profiles would seem to be safely in the realm of
fair use: http://en.wikipedia.org/wiki/Fair_use . The Google Books Ngrams
data, distributed up to 12-grams by Google, are one example of people
distributing rather high-N ngrams. Of course Google fought with the Authors
Guild over Google Books in general, but I don't recall this distribution of
ngram data being part of their fight, and in the end the Authors Guild
didn't win the obscurity they were pleading for. For another example,
http://commoncrawl.org/ distributes a massive crawl of the web for
researchers (or anybody) which is far more wholesale copying+redistribution
than you're proposing, but they follow the normal rules that webcrawlers
follow and are doing just fine (and are a very useful resource!).

So I would personally have zero legal worry about what you're proposing. I
would have no qualms about either academic research or commercial
applications (or commercial distribution) of that derived data. Adam is (as
usual) right, that you shouldn't even ask anybody for permission.

The thing about fair use that can make university lawyers uncomfortable is
that it's an "affirmative defense" -- you can argue it in court if someone
sues you, but there's no guarantee that you can use it to stay out of court
in the first place, which can be expensive.

But the other thing about the fair use defense is that, in order for you to
use it, somebody needs to be able to claim that you're infringing their
copyright in the first place. If you're just distributing frequency lists,
there's no trace of a copyrighted work to be found; even at the 5-gram
level, it's very hard to find any actionable infringement: the fourth
principle to be considered in evaluating fair use is "the effect of the use
upon the potential market for or value of the copyrighted work" and in your
case that effect should be exactly nil.

You could save yourself a bit of busywork, and maybe offer your
university's lawyers some psychological insulation from legal risk, by
using existing corpora resources like Common Crawl.

Part C of your question --- "are there jurisdictions where this might be
illegal" --- is the fuzziest to answer; the Berne Convention allows
signatory countries to define fair use for themselves, so there might be
jurisdictions where this could be risky, but they're probably places for
which it's challenging to get a visa anyway. I am not a lawyer, just a
copyright geek and a subscriber of "5 Useful Articles" by Parker Higgins &
Sarah Jeong, http://tinyletter.com/5ua , an amusing and edifying weekly
email about the inherent comedy of US IP law in the 21st century.

Cheers,

Orion

On Mon, Jan 5, 2015 at 9:30 PM, Adam Kilgarriff <
adam.kilgarriff at sketchengine.co.uk> wrote:

> Dear Damir,
>
> a few thoughts:
>
> In an innocent world view, the law says what is allowed and what is not.
> The more I see of how the legal profession works, the clearer it is that
> it's all political, in the sense that the judgements that build the case
> law (at least in UK) are made based on how well the lawyers played the
> game, how much money was involved, who had a sniff of how much money they
> might make.
>
> It's not about what is legal (which is always, in this area,
> underspecified), it is about risk management.
>
> If no-one sees a money-making opportunity, there is very little legal risk
> since no-one will take you to court.
> If you're a big organisation, you can always be taken to court and sued
> for large sums.  This has had horrible consequences for the JISC group at
> ISPRA: they are part of the EU, a very large organisation, and have had
> their work restricted by ambulance-chasing lawyers with a glint in their
> eyes for winning plump settlements.
>
> What you might be willing to do personally - given that you are probably,
> not, as an individual, worth suing, and your motivation for doing
> interesting work is high - is very different to what a (probably) rich
> organisation like your university might be willing to do.  If you want to
> do something, don't ask! (Specially not the university lawyers.  You'll
> probably never get an answer - even more frustrating than a simple 'no'.)
>
> Sorry if that is not very helpful
>
> Adam
>
>
> On 6 January 2015 at 04:00, Damir Cavar <dcavar at me.com> wrote:
>
>> Hi everybody,
>>
>> I know, this question has been addressed a lot, but, just to get an
>> update on this issue and your expert opinion:
>>
>> If I am accessing the internet from the US, as I am right now, and I
>> decide to generate N-gram-based language models by exploiting the web as
>> a corpus and publish the word-lists and frequency profiles openly on my
>> homepage, sell them even, change or manipulate them, and reuse them in
>> various ways, would this be
>>
>> a. ok as fair-use for research only, excluding commercial use
>> b. legal in general, independent of my research interests
>> c. legal only in some countries (so, my models would be illegal in some
>> others)
>>
>> What is the current status of the web as a corpus and extracted language
>> models from the legal perspective in the US and globally?
>>
>> If I do the same now with open-access journals and extract frequency
>> profiles of tokens for a certain research domain, would it be the same?
>> It I use Google Books? Or even some news website?
>>
>> Is the extraction of a language model, maybe a domain specific frequency
>> profile a copyright infringement per se? The text cannot be
>> reconstructed, the content is not visible, the authors style neither, in
>> particular not, if the corpus is larger etc.
>>
>> Thanks!
>>
>> Damir
>>
>>
>>
>> --
>> Damir Cavar
>> Department of Linguistics
>> Indiana University
>>
>>
>>
>> _______________________________________________
>> UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
>> Corpora mailing list
>> Corpora at uib.no
>> http://mailman.uib.no/listinfo/corpora
>>
>>
>
>
> --
> =============================================
> Adam Kilgarriff <http://www.kilgarriff.co.uk/>
> adam at sketchengine.co.uk
> Director                                    Lexical Computing Ltd
> <http://www.sketchengine.co.uk/>
> Visiting Research Fellow                 University of Leeds
> <http://leeds.ac.uk/>
> *Corpora for all* with the Sketch Engine <http://www.sketchengine.co.uk/>
>  and      SKELL <http://skell.sketchengine.co.uk/>
> =============================================
>
> _______________________________________________
> UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
> Corpora mailing list
> Corpora at uib.no
> http://mailman.uib.no/listinfo/corpora
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/corpora/attachments/20150105/6d2c2295/attachment.htm>
-------------- next part --------------
_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora