[Lexicog] FieldWorks Language Explorer V1 released

Wed Nov 8 08:46:15 UTC 2006

 FLEx* 1.0 is now ready for download as part of FieldWorks 4.0. 

The download options are found on: 

HYPERLINK
"http://www.sil.org/computing/fieldworks/FW_downloads.htm_"http://www.sil.or
g/computing/FieldWorks/FW_downloads.htm 

* a.k.a. Language Explorer

SIL International has just released version 1.0 of the new FieldWorks
Language Explorer (affectionately known as FLEx). It is part of the
FieldWorks suite of programs, which provide an integrated system for doing
linguistic and anthropological field work. The current version of FieldWorks
as a whole is 4.0. FLEx is designed to create and manage a dictionary,
create and maintain a text corpus, interlinearize texts, and study
morphology.

FLEx has a user interface that is similar to Shoebox and Toolbox with an
edit view for editing a single entry and a browse view for displaying the
database in a chart format. FLEx is similar to LinguaLinks in that it is an
object-oriented, relational database. This allows such features as
bi-directional links between entries. By way of contrast Shoebox only allows
one directional pointers in cross-reference fields. In Shoebox, you can add
a cross-reference field to an entry, but there is no guarantee that the
other entry actually exists. If you want to create reciprocal pointers, you
have to add a field to both entries. In FLEx, in a single step the user
links the two entries together, thus ensuring that both entries exist and
creating a cross-reference in both entries. There are many other advantages
to object-oriented, relational databases, such as data normalization. For
instance you can change the abbreviation for ‘pronoun’ from ‘pron’ to ‘prn’
in one place and it will be consistently changed everywhere in the project.

One disadvantage to an object-oriented, relational database is that the user
cannot directly manipulate the data. With Shoebox you could load the
database into a text editor or run a CC table on it. This was risky, because
you could easily corrupt the database. But many users liked this freedom. To
compensate for this lack, FLEx has very powerful built-in editing tools. You
can edit some fields in Browse View. You can sort fields from the start or
the end of the word(s). You have powerful filtering tools, including the
ability to use Regular Expressions in filters. The editing tools enable you
to make systematic changes to a field throughout the database. You have a
Find/Replace tool. You can copy the contents of one field to another. You
can copy a single word from one field to another with a single click of the
mouse, for instance to create a single word gloss from your definition
field. You can run a CC table on a field and see what the table will do
before the changes are applied. For the more technically oriented, you can
also run an ICU Unicode to Unicode transducer, TECkit, or Regular Expression
on a field. For instance these tools enable you to copy the contents of the
Lexeme Form field into the Pronunciation field and convert the orthographic
characters into IPA, all in a single step. The editing tools have a preview
feature that enables you to see what changes are going to be made and
enables you to confirm the change on a field by field basis. The advantage
of these tools is that you have the same power that you used to have, but
without the risk of mangling your database. In fact these tools are so
powerful and easy to use that you can develop your database in a fraction of
the time you could in other dictionary software programs, and do so with far
greater consistency and accuracy. (I will admit that I may be a bit
prejudiced in my opinion, since I helped design the tools.)

One of the best features of FLEx is the distinction it makes between data
and presentation. FLEx is designed to capture lexical data in a standardized
structure, but to present it for view or publication in a variety of
formats. The structure of lexical data has been hard-wired into the program.
In contrast Shoebox enforced no structure on the data (other than Standard
Format Markers), resulting in databases with a variety of ad hoc structures
and ad hoc fields. MDF was created to help solve this problem. Although it
provided a standard, the user was not forced to maintain the standard. In
fact few people are rigorous enough to maintain the standard even when they
try their hardest. We all make typing mistakes. This problem is now a thing
of the past, because FLEx provides a built-in structure. The user no longer
needs to worry about keeping the fields in the right order and is free to
concentrate on the data itself. All fields are available in all entries, but
you can decide which fields you want to see in each view.

The designers of the program have worked hard to create a theoretically
sound structure. Does this mean that you have no freedom to create
specialized dictionaries? Yes and no. By distinguishing between data and
presentation, FLEx enables you to customize which fields get printed and how
they will look. You can also create user-defined fields to handle special
needs. This enables you to publish a variety of products from a single
database. For instance you can create a monolingual dictionary, bilingual
dictionary, and a trilingual vocabulary, all from the same database. But
FLEx is not an unconstrained database manager like Shoebox. You cannot
maintain your address list in FLEx like you can in Shoebox. FLEx is designed
to describe a single language, although it can handle dialectal variants. It
is not designed for creating such things as comparative dictionaries for
many languages. For special data structures you need to use an unconstrained
database manager like Shoebox. But by standardizing the data structure, FLEx
can achieve far more power than it could if the data were unconstrained.
Paradoxically, constraints result in increased freedom and power.

FLEx was designed with multiple languages and multiple scripts in mind. All
data is Unicode. It supports left to right and right to left writing
systems, including complex non-Roman scripts. It supports multiple scripts
for a single language. It supports multiple analysis languages.

FLEx has other nice features. In the Lexicon Edit view (for editing a single
entry) there is a dictionary preview pane that shows how the entry will look
in print. There is a Dictionary view that shows how the entire dictionary
will look in print. (You can modify how the data will actually print, but
this gives an idea how it could look.) There is a Reversal Indexes view that
shows how the reversal index (finder list) will look in print. You can
produce an index for each analysis language. FLEx is not designed to create
full L1-L2 and L2-L1 dictionaries within the same project. To do this you
would need to set up two projects and display them in separate windows.

There is a Categorized Entry tool that enables you to efficiently enter
words that have been collected using the Dictionary Development Process
(DDP) list of semantic domains. In fact DDP has been integrated into FLEx
for those wishing to use that approach. There is also a Classified
Dictionary view that shows how the classified dictionary will look in print
(for those who have used DDP to classify their dictionary). This is like a
thesaurus, but also shows the grammatical category and definition of each
word. In this version of FLEx, some of the views appear to constrain what is
possible with the data structure. Other dictionary views and more
configuration options will be available in subsequent versions.

FLEx enables you to create a text corpus, either by typing texts directly
into the program, or by cutting and pasting them into an empty text record.
The program automatically generates a list of unique word forms from the
text corpus. There is a concordance feature that displays each occurrence of
a word form from the text corpus in column view, along with the wider
context of a selected occurrence. There is a built-in parser that can be
used to analyze a single word form or to facilitate interlinearizing texts.
The interlinearizer currently only handles the analysis of single words, but
plans are being made to allow the interlinearization of idioms and other
multi-word lexical items.

Many of the features that have been planned for FLEx have not yet been
implemented. However the program is robust and stable enough to warrant
release as version 1. Because FLEx is still under development, you will
encounter an occasional crash. However the program saves everything you do,
so you do not lose data when the occasional crash does occur. In my opinion,
FLEx has many advantages over Shoebox, LinguaLinks, or LexiquePro for
developing a dictionary. As the programmers continue to add new features, it
can only get better.

FLEx is currently a bit slow in switching from one view to another. This
problem is gradually diminishing as the programmers solve problems and
optimize performance. This slowness is more than compensated for by the
speed of the editing tools. The tools enable you to add, develop, and edit
fields at speeds 10 times, 100 times, and sometimes 1,000 times faster than
you could in any other program.

Importing data from Shoebox/Toolbox or LinguaLinks is a bit tricky. It is
necessary to get your Shoebox data into MDF format before importing it. This
can be a problem if you have not stuck with MDF. Finding and fixing the
errors in a Shoebox database can take anywhere from several hours to several
days, depending on the number and nature of the errors and your computer
skills. (These errors should be fixed anyway, since they will cause errors
when printing with MDF or LexiquePro.) Since FieldWorks requires Unicode,
you may also need to run some kind of conversion tool on your data to
convert it to Unicode. Fortunately FieldWorks comes with some standard
converters.

Currently FLEx does not have a print function, but you can export your data
in several formats. Exporting your data to LexiquePro requires you to go
through the LexiquePro import process as you would with any database. This
took me several minutes. You only need to do this once. After that you can
continue editing in FLEx. It is then a simple step to re-export your
dictionary from FLEx and load it into LexiquePro.

In the last five months I’ve taught three lexicography workshops using
FieldWorks. (The last workshop was videoed and will be available soon.) I
received some very positive feedback concerning the program. One
lexicography consultant said, “This is the way to do dictionaries.” A
linguistics consultant said, “There is no way I would go back to LinguaLinks
or Shoebox.”

The best feature of FieldWorks is that it is absolutely free! You can learn
more and download the program from the SIL website: HYPERLINK
"http://www.sil.org/computing/fieldworks/index.html"http://www.sil.org/compu
ting/fieldworks/index.html

Ron Moe

SIL Linguistics Consultant

(Developer of the Dictionary Development Process and one of the lexicography
consultants for the FieldWorks team)

   _____  

--
No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.1.409 / Virus Database: 268.13.27/517 - Release Date: 11/3/2006

-- 
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.409 / Virus Database: 268.14.0/524 - Release Date: 11/8/2006

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/lexicography/attachments/20061108/6dbf3945/attachment.htm>