Release for 1.7.

git-svn-id: file:///home/svnrepos/software/liblbfgs/trunk@56 ecf4c44f-38d1-4fa4-9757-a0b4dd0349fc

COPYING

@@ -1,7 +1,7 @@
 The MIT License
 
 Copyright (c) 1990 Jorge Nocedal
-Copyright (c) 2007,2008 Naoaki Okazaki
+Copyright (c) 2007-2009 Naoaki Okazaki
 
 Permission is hereby granted, free of charge, to any person obtaining a
 copy of this software and associated documentation files (the "Software"),

ChangeLog

@@ -1,3 +1,35 @@
+2009-02-28  Naoaki Okazaki  <okazaki at chokkan org>
+
+	* libLBFGS 1.7:
+	- Improved OWL-QN routines for stability.
+	- Removed the support of OWL-QN method in MoreThuente algorithm
+	  because it accidentally fails in early stages of iterations for some
+	  objectives. Because of this change, the OW-LQN method must be used
+	  with the backtracking algorithm (LBFGS_LINESEARCH_BACKTRACKING), or
+	  the library returns LBFGSERR_INVALID_LINESEARCH.
+	- Renamed line search algorithms as follows:
+	    - LBFGS_LINESEARCH_BACKTRACKING: regular Wolfe condition.
+	    - LBFGS_LINESEARCH_BACKTRACKING_LOOSE: regular Wolfe condition.
+	    - LBFGS_LINESEARCH_BACKTRACKING_STRONG: strong Wolfe condition.
+	- Source code clean-up.
+
+
+2008-11-02  Naoaki Okazaki  <okazaki at chokkan org>
+
+	* libLBFGS 1.6:
+	- Improved line-search algorithm with strong Wolfe condition, which
+	  was contributed by Takashi Imamichi. This routine is now default for
+	  LBFGS_LINESEARCH_BACKTRACKING. The previous line search algorithm
+	  with regular Wolfe condition is still available as
+	  LBFGS_LINESEARCH_BACKTRACKING_LOOSE.
+	- Configurable stop index for L1-norm computation. A member variable
+	  lbfgs_parameter_t::orthantwise_end was added to specify the index
+	  number at which the library stops computing the L1 norm of the
+	  variables. This is useful to prevent some variables from being
+	  regularized by the OW-LQN method.
+	- A sample program written in C++ (sample/sample.cpp).
+
+
 2008-07-10  Naoaki Okazaki  <okazaki at chokkan org>
 
 	* libLBFGS 1.5:

README

@@ -2,7 +2,7 @@
            libLBFGS: C library of limited-memory BFGS (L-BFGS)
 
                                        Copyright (c) 1990, Jorge Nocedal 
-                                 Copyright (c) 2007,2008, Naoaki Okazaki 
+                                 Copyright (c) 2007-2009, Naoaki Okazaki 
 
 =========================================================================
 1. Introduction

configure.in

@@ -18,7 +18,7 @@ AC_CONFIG_SRCDIR([lib/lbfgs.c])
 dnl ------------------------------------------------------------------
 dnl Initialization for automake
 dnl ------------------------------------------------------------------
-AM_INIT_AUTOMAKE(liblbfgs, 1.6)
+AM_INIT_AUTOMAKE(liblbfgs, 1.7)
 AC_CONFIG_HEADERS(config.h)
 AM_MAINTAINER_MODE
 

include/lbfgs.h

@@ -153,9 +153,11 @@ enum {
     /** MoreThuente method proposd by More and Thuente. */
     LBFGS_LINESEARCH_MORETHUENTE = 0,
     /** Backtracking method with strong Wolfe condition. */
-    LBFGS_LINESEARCH_BACKTRACKING_STRONG,
+    LBFGS_LINESEARCH_BACKTRACKING_STRONG = 1,
     /** Backtracking method with regular Wolfe condition. */
-    LBFGS_LINESEARCH_BACKTRACKING,
+    LBFGS_LINESEARCH_BACKTRACKING = 2,
+    /** Backtracking method with regular Wolfe condition. */
+    LBFGS_LINESEARCH_BACKTRACKING_LOOSE = 2,
 };
 
 /**
@@ -184,7 +186,25 @@ typedef struct {
      */
     lbfgsfloatval_t epsilon;
 
+    /**
+     * Distance for delta-based convergence test.
+     *  This parameter determines the distance, in iterations, to compute
+     *  the rate of decrease of the objective function. If the value of this
+     *  parameter is zero, the library does not perform the delta-based
+     *  convergence test. The default value is \c 0.
+     */
     int             past;
+
+    /**
+     * Delta for convergence test.
+     *  This parameter determines the minimum rate of decrease of the
+     *  objective function. The library stops iterations when the
+     *  following condition is met:
+     *      (f' - f) / f < \ref delta,
+     *  where f' is the objective value of \ref past iterations ago, and f is
+     *  the objective value of the current iteration.
+     *  The default value is \c 0.
+     */
     lbfgsfloatval_t delta;
 
     /**
@@ -529,18 +549,30 @@ This library is used by:
 
 @section download Download
 
-- <a href="http://www.chokkan.org/software/dist/liblbfgs-1.6.tar.gz">Source code</a>
+- <a href="http://www.chokkan.org/software/dist/liblbfgs-1.7.tar.gz">Source code</a>
 
 libLBFGS is distributed under the term of the
 <a href="http://opensource.org/licenses/mit-license.php">MIT license</a>.
 
 @section changelog History
+- Version 1.7 (2009-02-28):
+    - Improved OWL-QN routines for stability.
+    - Removed the support of OWL-QN method in MoreThuente algorithm because
+      it accidentally fails in early stages of iterations for some objectives.
+      Because of this change, <b>the OW-LQN method must be used with the
+      backtracking algorithm (::LBFGS_LINESEARCH_BACKTRACKING)</b>, or the
+      library returns ::LBFGSERR_INVALID_LINESEARCH.
+    - Renamed line search algorithms as follows:
+        - ::LBFGS_LINESEARCH_BACKTRACKING: regular Wolfe condition.
+        - ::LBFGS_LINESEARCH_BACKTRACKING_LOOSE: regular Wolfe condition.
+        - ::LBFGS_LINESEARCH_BACKTRACKING_STRONG: strong Wolfe condition.
+    - Source code clean-up.
 - Version 1.6 (2008-11-02):
     - Improved line-search algorithm with strong Wolfe condition, which was
       contributed by Takashi Imamichi. This routine is now default for
-      ::LBFGS_LINESEARCH_BACKTRACKING_STRONG. The previous line search algorithm
+      ::LBFGS_LINESEARCH_BACKTRACKING. The previous line search algorithm
       with regular Wolfe condition is still available as
-      ::LBFGS_LINESEARCH_BACKTRACKING.
+      ::LBFGS_LINESEARCH_BACKTRACKING_LOOSE.
     - Configurable stop index for L1-norm computation. A member variable
       ::lbfgs_parameter_t::orthantwise_end was added to specify the index
       number at which the library stops computing the L1 norm of the
@@ -552,7 +584,7 @@ libLBFGS is distributed under the term of the
       ::lbfgs_parameter_t::orthantwise_start was added to specify the index
       number from which the library computes the L1 norm of the variables.
       This is useful to prevent some variables from being regularized by the
-      OW-LQN method.
+      OWL-QN method.
     - Fixed a zero-division error when the initial variables have already
       been a minimizer (reported by Takashi Imamichi). In this case, the
       library returns ::LBFGS_ALREADY_MINIMIZED status code.
@@ -563,7 +595,7 @@ libLBFGS is distributed under the term of the
     - Configurable line search algorithms. A member variable
       ::lbfgs_parameter_t::linesearch was added to choose either MoreThuente
       method (::LBFGS_LINESEARCH_MORETHUENTE) or backtracking algorithm
-      (::LBFGS_LINESEARCH_BACKTRACKING_STRONG).
+      (::LBFGS_LINESEARCH_BACKTRACKING).
     - Fixed a bug: the previous version did not compute psuedo-gradients
       properly in the line search routines for OWL-QN. This bug might quit
       an iteration process too early when the OWL-QN routine was activated
@@ -626,6 +658,9 @@ method presented in:
       In <i>Proceedings of the 24th International Conference on Machine
       Learning (ICML 2007)</i>, pp. 33-40, 2007.
 
+Special thanks go to Yoshimasa Tsuruoka and Daisuke Okanohara for technical
+information about OWL-QN.
+
 Finally I would like to thank the original author, Jorge Nocedal, who has been
 distributing the effieicnt and explanatory implementation in an open source
 licence.
@@ -633,7 +668,7 @@ licence.
 @section reference Reference
 
 - <a href="http://www.ece.northwestern.edu/~nocedal/lbfgs.html">L-BFGS</a> by Jorge Nocedal.
-- <a href="http://research.microsoft.com/research/downloads/Details/3f1840b2-dbb3-45e5-91b0-5ecd94bb73cf/Details.aspx">OWL-QN</a> by Galen Andrew.
+- <a href="http://research.microsoft.com/en-us/downloads/b1eb1016-1738-4bd5-83a9-370c9d498a03/default.aspx">Orthant-Wise Limited-memory Quasi-Newton Optimizer for L1-regularized Objectives</a> by Galen Andrew.
 - <a href="http://chasen.org/~taku/software/misc/lbfgs/">C port (via f2c)</a> by Taku Kudo.
 - <a href="http://www.alglib.net/optimization/lbfgs.php">C#/C++/Delphi/VisualBasic6 port</a> in ALGLIB.
 - <a href="http://cctbx.sourceforge.net/">Computational Crystallography Toolbox</a> includes

sample/sample.c

@@ -65,7 +65,7 @@ int main(int argc, char *argv)
     /* Initialize the parameters for the L-BFGS optimization. */
     lbfgs_parameter_init(&param);
     //param.orthantwise_c = 1;
-    param.linesearch = LBFGS_LINESEARCH_BACKTRACKING_STRONG;
+    //param.linesearch = LBFGS_LINESEARCH_BACKTRACKING;
 
     /*
         Start the L-BFGS optimization; this will invoke the callback functions