Thursday, March 31, 2016

]On Discriminative vs. Generative classifiers: A comparison ... posterior probabilities needed for classifi- cation,

4.1.3 Knowledge Combination The HMM structure makes strong independence assumptions: (1) that features depend only on the current state (and in practice, as we saw, only on the event label) and (2) that each word+event label depends only on the last N 1 tokens. In return, we get a computationally efficient structure that allows information from the entire sequence W; F to inform the posterior probabilities needed for classifi- cation, via the forward-backward algorithm. More problematic in practice is the integration of multiple word-level features, such as POS tags and chunker output. Theoretically, all tags could simply be included in the hidden state representation to allow joint modeling of words, tags, and events. However, this would drastically increase the size of the state space, making robust model estimation with standard N-gram techniques difficult. A method that works well in practice is linear interpolation, whereby the conditional probability estimates of various models are simply averaged, thus reducing variance


PDF]On Discriminative vs. Generative classifiers: A comparison ...

ai.stanford.edu/~ang/.../nips01-discriminativegenerative....
Stanford AI Lab
by AY Ng - ‎Cited by 1324 - ‎Related articles
On Discriminative vsGenerative classifiers: A comparison of logistic regression and naive Bayes. Andrew Y. Ng. Computer Science Division. University of ...

What is the difference between a Generative and ...

stackoverflow.com/.../what-is-the-difference-between-a-generative-and-d...
May 18, 2009 - This paper is a very popular reference on the subject of discriminative vsgenerative classifiers, but it's pretty heavy going. The overall gist is ...

bayesian inference

maximize the negative of the actual objective function , The Gradient Projection Method for Nonlinear Programming. Part I. Linear Constraints J. B. Rosen

The Gradient Projection Method for Nonlinear Programming ...

https://www.jstor.org/stable/10.2307/2098960
JSTOR
convex (maximizing a concave) nonlinear objective function subject to ..... -lYm is the negative eigenvalue with largest absolute value of C(x) for x in R. Using the ...

[PDF]Constrained Optimization 5 - Mechanical and Aerospace ...

www2.mae.ufl.edu/nkim/eas6939/ConstrainedOpt.pdf
function subject to equality and inequality constraints minimize f(x) such that hi(x)=0, ... with the minimum value of the functions more closely. The value of .... direction with a negative slope for the objective function that does not violate the ...... [9] Rosen, J.B., “The Gradient Projection Method for Nonlinear Programming—Part.

Constrained optimization along geodesics - ScienceDirect

www.sciencedirect.com/science/.../pii/0022247X81900263
ScienceDirect
by CA Botsaris - ‎1981 - ‎Cited by 16 - ‎Related articles
... to the geodesic starting from x1' in the direction of the projected negative gradient at .... Sufficient descent in the value of the objective function is achieved by scaling ... Find the smallest local minimizer r^ of minimize: f(xk + s{xk, t)), t > 0, where s(;c*, .... J. ROSEN, The gradient projection method for nonlinear programming.

An algorithm for linearly constrained nonlinear programming ...

www.sciencedirect.com/science/.../pii/0022247X81900470
ScienceDirect
by MS Bazaraa - ‎1981 - ‎Cited by 1 - ‎Related articles
The former computes a direction by projecting the negative gradient on the space ... based on a quadratic approximation to the objective function is computed, and then .... Since I(xi) c I(x*), then the optimal objective value g'(Xt) to Problem u'(x,) ..... J. B. ROSEN, The gradient projection method for nonlinear programming.

[PDF]PROJECTED NEWTON METHODS FOR ... - MIT

www.mit.edu/.../ProjectedNewton....
Massachusetts Institute of Technology
by DP BERTSEKAS - ‎1982 - ‎Cited by 534 - ‎Related articles
Mar 2, 1982 - where f : Rn -, R is a continuously differentiable function, and the vector inequality .... minimize Vf(xk)' ( X - xk) +$(x - x ~ ) ' v ~ ~ ( x ~ ) ( x ..... In conclusion, the algorithm is well defined, decreases the value of the objective ...... [7] J. B. ROSEN, The gradient projection method for nonlinear programming, Part I: ...

[PDF]discussion paper no. 209 - Kellogg School of Management

www.kellogg.northwestern.edu/.../209.p...
Kellogg School of Management
by A Perry - ‎1976 - ‎Related articles
mation on the estimated partial derivatives of the objective function (1) . ... minimizeZ X .f(x.) ... where f(xj) is the actual value of f(x) at the design point xj, and gi(xj)v is the ... The negative weighted gradient of (15) is actually the first direction taken ... Rosen, J. , "The Gradient Projection Method for Nonlinear Programming, I.

[PS]Conjugate Gradient Projection Approach for Multi-Antenna ...

https://www.ece.vt.edu/thou/VT_TR_5.ps
by J Liu - ‎Cited by 12 - ‎Related articles
Maximize logdet (I + ∑K ... Due to the complexity of the objective function in (2), we adopt the inexact line ... If the maximum absolute value of the elements in Q .... the projection of D − µI onto the negative semidefinite cone. ..... [13] J. B. Rosen, “The gradient projection method for nonlinear programming, Part I, Linear con-.

[PDF]Rosen's Projection Method for SVM Training - UCL/ELEN

https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2009-123.pdf
by J López - ‎Cited by 3 - ‎Related articles
training seeks [1] to maximize the margin of a separating hyperplane by solving min. 1. 2 .... On the other hand, if α does not solve (3), there will be at least one negative .... iterations, as it automatically detects whether the objective function has been ... The gradient projection method for nonlinear programming, i: Linear con-.

[PDF]A New Projected Quasi-Newton Approach for the ...

ftp://ftp.cs.utexas.edu/pub/.../tr06-54.pdf
University of Texas at Austin
by D Kim - ‎Cited by 29 - ‎Related articles
or equivalently, as a problem over the free variables, minimize y gk(y) = 1. 2. ¯Ay − b2, subject to y .... Let m be the smallest non-negative integer for which ... a monotonic descent in the objective function value (Lemma 1 for APA, and Lemma 2 for LM). Then, ...... The Gradient Projection Method for Nonlinear Programming.

Full text of "A nonlinear programming algorithm for an array ...

https://archive.org/.../nonlinearprogram357mulv_djvu.t...
Internet Archive
Test for an optimum value and return to step 1 if the test fails. .... The problem is tominimize the objective function w = f (X) = x x + 2x 2 subject .... It attempts to move the solution in the direction of the negative gradient, similar ...... 181-217- [12] Rosen, J.B., "The Gradient Projection Method for Nonlinear Programming, Part II: ...

NLP optimization Rosen’s Gradient Projection Method [16].

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.6.5175&rep=rep1&type=pdf

Optimization on this function will make the inner-divergence described in the first two terms on the right hand side as small as possible while the inter-class divergence among classes will be as big as possible, which will benefit the classification greatly. Different from the discriminative classifiers such as the LR, the discriminative information is finally incorporated into the joint probability p1 and p2. Thus the advantages of using joint probabilities will be naturally inherited into the discriminative Naive Bayesian classifier. However, the disadvantage of adding into this interactive item is that we cannot optimize p1 and p2 as in NB separately in the sub-dataset 1 and sub-dataset 2. To clarify this problem, we combine the NB assumption to expand the optimization function into a complete form: min {p1,p2} X 2 c=1 Xn j=1 X Aj [pˆc(ajk)log ˆ pc(ajk) pc(ajk) ] + W · 1 Pn j=1 P Aj p1(ajk)log(p1(ajk)/p2(ajk)),(10) s.t. 0 ≤ pc(ajk) ≤ 1, (11) X Aj pc(ajk) = 1, c = 1, 2;j = 1, 2, . . . n. (12) pc(ajk) is the short form of pc(Aj = ajk). So does pˆc(ajk). p1 and p2 are a set of parameters, namely, p1 = {p1(Aj ), 1 ≤ j ≤ n}, p2 = {p2(Aj ), 1 ≤ j ≤ n}. This is a nonlinear optimization problem under linear constraints. p1 and p2 are interactive variables. It is clear that they cannot be separately optimized as in Eq.( 7). To solve this problem, we use a modified Rosen’s Gradient Projection Method [16]. We firstly calculate the gradient of the optimization function w.r.t p1 and p2 as Eq. (13). We then project this gradient on the constraint plane. In our problem the projection matrix can be written as Eq. (16). The optimal step length α is searched in the projected gradient direction by using Quadratic Interpolation method [11]. The process is repeated until a local minimal is obtained. We write down the detailed steps as follows:

Gradient Projection Method for Nonlinear Programming

docs.lib.purdue.edu/cgi/viewcontent.cgi?article...

Purdue University
by LD Pyle - ‎1971 - ‎Cited by 1 - ‎Related articles
Jun 10, 1971 - Pyle, L. Duane, "A Simplex Algorithm - Gradient Projection Method for Nonlinear Programming" (1971). ... Frisch and Rosen are based on an interestingmethod for inverting ..... for example, by executing Phase I of the simplex.

The Gradient Projection Method for Nonlinear Programming ...

www.jstor.org/stable/2098960
JSTOR
by JB Rosen - ‎1960 - ‎Cited by 1256 - ‎Related articles
the gradient projection method will also solve a linear programming prob- lem. In Part II of the paper ... 182 J. B. ROSEN in 1956 [15]. A number ... GRADIENT PROJECTION METHOD OF NONLINEAR PROGRAMMING 183 projection method ...

[PDF]Gradient Projection Method for Nonlinear Programming

docs.lib.purdue.edu/cgi/viewcontent.cgi?article...
Purdue University
by LD Pyle - ‎1971 - ‎Cited by 1 - ‎Related articles
Jun 10, 1971 - ABSTRACT. Witzgall [ 7 L commenting on the gradient projection methods ... is generated using the simplex algorithm, whereas Rosen gives a.

On Rosen's gradient projection methods - Springer

link.springer.com/.../10.1007%2FBF02...
Springer Science+Business Media
by DZ Du - ‎1990 - ‎Cited by 24 - ‎Related articles
This paper is a survey of Rosen's projection methods in nonlinear programming. Through the discussion of previous works, we propose some interesting ...

The Gradient Projection Method for Nonlinear Programming ...

epubs.siam.org/.../0108011
Society for Industrial and Applied Mathematics
by JB Rosen - ‎1960 - ‎Cited by 1256 - ‎Related articles
The Gradient Projection Method for Nonlinear Programming. Part I. ... J. B. Rosen... (1982) Projected Newton Methods for Optimization Problems with Simple ...

[PDF]Chapter 5: Constrained Optimization 5.5 Gradient Projection ...

www2.esm.vt.edu/~zgurdal/COURSES/4084/4084-Docs/.../GradProj.pdf
Rosen's gradient projection method is based on projecting the search .... Thegradient projection method has been generalized by Rosen to nonlinear con-.

The Gradient Projection Method for Non-Linear ...

https://www.researchgate.net/.../250956400_The_Gradient_Pr...
ResearchGate
The Gradient Projection Method for Non-Linear Programming, Part II. Non-Linear Constraints on ResearchGate, the professional network for ... J. B. Rosen.

The Gradient Projection Method for Non-Linear ...

https://www.researchgate.net/.../240430418_The_Gradient_Pr...
ResearchGate
The Gradient Projection Method for Non-Linear Programming, Part I. Linear ... which can be solved by using the gradient projection (GP) algorithm (Rosen, ...

[PDF]Constrained Optimization 5 - Mechanical and Aerospace ...

www2.mae.ufl.edu/nkim/eas6939/ConstrainedOpt.pdf
Most problems in structural optimization must be formulated as constrained min ...... The gradient projection method has been generalized by Rosen to nonlinear ...

[PPT]Engineering Optimization

www.zfm.ethz.ch/e/v/opt/handouts/ETHZ_Lecture10.ppt
ETH Zurich
Engineering Optimization – Concepts and Applications ... Rosen's gradient projection method; Zoutendijk's method of feasible directions ... nonlinearconstraints).

[PDF]CONSTRAINED NONLINEAR PROGRAMMING

www.pitt.edu/~jrclass/opt/notes4.pdf
University of Pittsburgh
We now turn to methods for general constrained nonlinear programming. These may ...... Rosen's method works by projecting -∇f(x) on to the .... The gradient projectionalgorithm can be generalized to nonlinear constraints by projecting the ...