Saturday, October 22, 2016

TravelSky Technology(China Airline): Predicting Airfare Price by SVM

TravelSky Technology(China Airline):  Predicting Airfare Price by SVM


We were faced with the large dataset with no explicit links between records, making it a very challenging task to analyze price changes of an individual round-trip.
It was much more practical to develop a model that generalizes the properties of all records in the dataset, and to train a SVM as a binary pricing classifier to distinguish between ”expensive” and ”cheap” of all tickets (transaction records) processed. 

Part I. General Introduction
Travelsky technologies is one of the largest global distribution system in the travel/tourism industry: it sells tickets for all airlines (also hotels) and processes millions billable transactions per month.

Project Goals

1 Construct and train a general classifier so that it can distinguish between expensive and cheap tickets.

2. Use this classifier to predict the prices of future tickets.

3. Determine which factors have the greatest impact on price by analyzing the trained classifier.

Exploratory data analysis
Extent of the dataset:  billion records, 132.2 GiB (uncompressed) , hundreds  departure airports, hundreds destinations, hundreds routes, hundreds airlines.

Lots of fields:  “Buy” date: When was this price current? “Fly” date: When does the flight leave? Price.  Cabin class Economy/Business/First (98% economy tickets) . Booking class A-Z  … Airline The airline selling the ticket. some data looks like a time series, tickets are linked over time

Classification & Prediction methods

Implemented two different classifiers:  Support vector machine (SVM),  L1- regularized linear regression.  Both are convex minimization problems that can be solved online by employing the stochastic gradient descent (SGD) method.

 SVM: binary linear classifier. Goal: Find maximum-margin hyperplane that divides the points with label y “+1” from those with label y “-1”. 
Training: Generate training label yi for i-th data point xi.  Choose hyperplane parameters so the margin is maximal and the training data is still correctly classified.

Preprocessing
For each route r, calculate the arithmetic mean (and standard deviation) of the price over all tickets. 
Assign labels: Label +: “Above mean price for this route”.  Label -: “Below mean price for this route” .Only store mean/std-dev, do not actually store labels.

Feature Selection
Extract features from plaintext records (x).
Each plaintext record is transformed into a 990-dimensional vector.
Each dimension contains a numerical value corresponding to a feature such as:  Number of days between “Buy” and “Fly” dates, Week of day (for all dates) , Is the day on a weekend (for all dates), Dates isMonday, isWeekend, isWinter, weekOfYear, . . . .
Each dimension is normalized to zero mean and unit variance (per route r).

Part II. More Detailed Descriptions of Our Model

Classification methods


In order to identify which records represent cheap tickets and which records have traits identifying them as expensive tickets, a classifier able to distinguish between ”expensive” and ”cheap” records is necessary.

It should be possible to train such a classifier on all records at once, identifying the features making a record cheaper or more expensive than other records. As some routes are more expensive than others, it does not make sense to include the route as a feature, but rather normalize prices per route. This enables comparison of prices across all routes without simply marking all records of a particular route as expensive. Each record is then labeled according to the normalized price.
In short, a record for a particular route is labeled as ”expensive” (+1) if its price is higher than the average price of all records for that specific route. Otherwise it is labeled as ”cheap” (-1).
After training the classifier, it should be able to predict a label from  and assign this label to a new record with an unknown price. As the route of the new record is known, a numerical minimal or maximal price (the afore-mentioned average price per route) can be directly inferred from the predicted label.
Additionally, the model parameters of the trained classifier should contain information on how much each feature contributes to a record being cheap or expensive.

Online algorithms for classification

Due to the large amount of data, algorithms using more than a constant amount of memory are not suitable. Two algorithms were implemented, one for online support vector machines and the other for online-regularized logistic regression. This allows efficient training of the classifier on a parallel system with limited memory.

Some definitions and terminology as they are used in the following sections:

Xi
Each data point Xi  represents the features of a single record Ri and is also called the feature
vector for record Ri
Each component contains information about a single aspect of the record Ri . The contents are described previously are derived from the fields of Ri


Yi

Each label yi represents the label (classification) of a single record Ri, with two values: ”expensive” (+1), “cheap” (-1).

Record Ri always consists of a pair (Xi Yi) , both values are known for trading dataset.
For new data points Xi,  the value of the labels Yi is initially unknown,  and is the result of the classification/prediction. A label has only two possible values: −1 and 1.

W

The weight vector w is the model parameter of the classifier to be estimated and is initially

unknown. In both classifiers discussed below, w has the same number of dimensions as a data

point Xi and determines the effect each value in Xi has on the classification result.

Feature vector generation
For each record, a feature vector consisting of 990 features was created. Before normalization, each entry was set to either 1.0 (boolean true), 0 (boolean false) or a value associated with a numerical field in the record.
The feature vector represents each record as a 990-dimensional vector.
Some examples of features, and record fields utilized:
Dates Request Date, Departure Date, Return Date
Date differences Return-Departure Date, Departure-Request Date
Categorical values Passenger Type, Airline
Numerical values Number of passengers, Number of hops
Sequences of categorical values Cabin Classes, Booking Classes, Availabilities
Sequences of numerical values Flight numbers
Feature vector normalization
As with price normalization, each of the 990 features fm was normalized in two steps. In a first MapReduce job, the arithmetic means µfm and standard deviations σfm were calculated using the same methods as for price normalization and subsequently stored to disk.
All following MapReduce jobs loaded the 990 means and standard deviations from disk and calculated the normalized feature vector x 0 i on-the-fly by calculating the standard score of each feature fm: f 0 m = fm − µfm σfm , m 1, . . . , 990

Stochastic gradient descent (SGD)

Given a convex set S and a convex function f, we can estimate the parameter w in min

f(w) is of the form f(w) = Pn

a single observed data point from the dataset. Finding w is done iteratively, by using one random

sample data point from the dataset per iteration. For regularization, w S needs to be ensured,

thus a projection onto S is necessary.

Let w0 S be the starting value. Then each iteration t consists of the update step
t=1 ft(w). Usually, each summand ft represents the loss function for

wt+1 = P rojS(wt − ηtft(wt))

where P rojS is a projection onto the set S, ηt is the current step size (learning rate), and ft is the

gradient of f approximated at the sample data point for iteration t.

It is possible to only use a subsample of the full dataset if the data points used for training are picked


at random from the dataset. Training can then either be halted after a fixed number of iterations or as soon as sufficient accuracy is achieved.

Friday, May 6, 2016

Rule learner (or Rule Induction)

Rule learner (or Rule Induction)

It is also known as Separate-And-Conquer method. This method apply an iterative process consisting in first generating a rule that covers a subset of the training examples and then removing all examples covered by the rule from the training set. This process is repeated iteratively until there are no examples left to cover. The final rule set is the collection of the rules discovered at every iteration of the process [13]. Some examples of these kinds of systems are:
  • OneR
OneR or “One Rule” is a simple algorithm proposed by Holt. The OneR builds one rule for each attribute in the training data and then selects the rule with the smallest error rate as its ‘one rule’. To create a rule for an attribute, the most frequent class for each attribute value must be determined. The most frequent class is simply the class that appears most often for that attribute value. A rule is simply a set of attribute values bound to their majority class. OneR selects the rule with the lowest error rate. In the event that two or more rules have the same error rate, the rule is chosen at random.
R.C. Holte (1993). Very simple classification rules perform well on most commonly used datasets. Machine Learning. 11:63-91.
  • Ridor
Ridor algorithm is the implementation of a RIpple-DOwn Rule learner proposed by Gaines and Compton. It generates a default rule first and then the exceptions for the default rule with the least (weighted) error rate. Then it generates the “best” exceptions for each exception and iterates until pure. Thus it performs a tree-like expansion of exceptions. The exceptions are a set of rules that predict classes other than the default. IREP is used to generate the exceptions.
Brian R. Gaines, Paul Compton (1995). Induction of Ripple-Down Rules Applied to Modeling Large Databases. J. Intell. Inf. Syst.. 5(3):211-228.
  • PART
PART is a separate-and-conquer rule learner proposed by Eibe and Witten. The algorithm producing sets of rules called ‘decision lists’ which are ordered set of rules. A new data is compared to each rule in the list in turn, and the item is assigned the category of the first matching rule (a default is applied if no rule successfully matches). PART builds a partial C4.5 decision tree in each iteration and makes the “best” leaf into a rule. The algorithm is a combination of C4.5 and RIPPER rule learning.
Eibe Frank, Ian H. Witten: Generating Accurate Rule Sets Without Global Optimization. In: Fifteenth International Conference on Machine Learning, 144-151, 1998.
  • JRip (RIPPER)
JRip implements a propositional rule learner, Repeated Incremental Pruning to Produce Error Reduction (RIPPER), which was proposed by William W. Cohen as an optimized version of IREP. Ripper builds a ruleset by repeatedly adding rules to an empty ruleset until all positive examples are covered. Rules are formed by greedily adding conditions to the antecedent of a rule (starting with empty antecendent) until no negative examples are covered. After a ruleset is constructed, an optimization postpass massages the ruleset so as to reduce its size and improve its fit to the training data. A combination of cross-validation and minimum-description length techniques is used to prevent overfitting.
Cohen, W. W. 1995. Fast effective rule induction. In Machine Learning: Proceedings of the Twelfth International Conference, Lake Tahoe, California.http://citeseer.ist.psu.edu/cohen95fast.html
  • DecisionTable
DecisionTable algorithm builds and using a simple decision table majority classifier as proposed by Kohavi. It summarizes the dataset with a ‘decision table’ which contains the same number of attributes as the original dataset. Then, a new data item is assigned a category by finding the line in the decision table that matches the non-class values of the data item. DecisionTable employs the wrapper method to find a good subset of attributes for inclusion in the table. By eliminating attributes that contribute little or nothing to a model of the dataset, the algorithm reduces the likelihood of over-fitting and creates a smaller and condensed decision table.
Ron Kohavi: The Power of Decision Tables. In: 8th European Conference on Machine Learning, 174-189, 1995.
  • ConjunctiveRule
ConjuctiveRule algorithm implements a single conjunctive rule learner that can predict for numeric and nominal class labels. A rule consists of antecedents “AND”ed together and the consequent (class value) for the classification/regression. In this case, the consequent is the distribution of the available classes (or mean for a numeric value) in the dataset. If the test instance is not covered by this rule, then it’s predicted using the default class distributions/value of the data not covered by the rule in the training data. This learner selects an antecedent by computing the Information Gain of each antecedent and prunes the generated rule using Reduced Error Pruning (REP) or simple pre-pruning based on the number of antecedents. For classification, the Information of one antecedent is the weighted average of the entropies of both the data covered and not covered by the rule.

5 Responses to Rule learner (or Rule Induction)

  1. Salaam,
    very nice blog.
    i found the summaries to b well written😉
  2. Ridz says:
    Thank you. You have a very interesting blog too. I will add it to my list:)
    Ridz
  3. Spyder says:
    Thank You. Excellently Summarized.
  4. Anonymous says:
    hello,
    nice blog brother
    I have a request please
    I need deep information about “JRip” for a presentation
    Can you help me to find good resources ?
  5. Manal says:
    hello,
    nice blog brother
    I have a request please
    I need deep information about “JRip” for a presentation
    Can you help me to find good resources ?

洛仑兹规范6_百度文库

wenku.baidu.com/.../5133dcfbaef8941ea76e0532.htm...
Translate this page
Apr 15, 2011 - 本文通过麦克斯韦方程组引入电磁场规范,指出库伦规范和洛仑兹规范只是众多电磁场规范中的两种较特殊的规范,最后推导出在静态场中库伦规范和 ...

库仑规范_图文_百度文库

wenku.baidu.com/.../cd58693631126edb6f1a1051.ht...
Translate this page
Nov 10, 2011 - 这是讲的比较清楚的一个,且还包含其他规范,推荐 .... 项对应库仑场E ,? ?t 对应着感应库 r 场E 。 感b) 洛仑兹规范(Lorentz gauge) 洛仑兹 

电磁辐射中, 需要根据激发源来决定电磁场的性质, 而麦克斯韦方程组用电场强度 E r 和电磁感应强度 B 对电磁场进行描述,与激发源没有直接的联系,难以直接的描述电磁场, 因此,为了能够由激发源直接描述电磁场,引入了势函数的概念。

洛仑兹规范6_百度文库

wenku.baidu.com/.../5133dcfbaef8941ea76e0532.htm...
Translate this page
Apr 15, 2011 - 本文通过麦克斯韦方程组引入电磁场规范,指出库伦规范和洛仑兹规范只是众多电磁场规范中的两种较特殊的规范,最后推导出在静态场中库伦规范和 ...

库仑规范_图文_百度文库

wenku.baidu.com/.../cd58693631126edb6f1a1051.ht...
Translate this page
Nov 10, 2011 - 这是讲的比较清楚的一个,且还包含其他规范,推荐 .... 项对应库仑场E ,? ?t 对应着感应库 r 场E 。 感b) 洛仑兹规范(Lorentz gauge) 洛仑兹 


洛仑兹规范6_自然科学_专业资料

暂无评价|0人阅读|0次下载|举报文档
洛仑兹规范6_自然科学_专业资料。本文通过麦克斯韦方程组引入电磁场规范 ,指出库伦规范和洛仑兹规范只是众多电磁场规范 中的两种较特殊的规范,最后推导出在静态场中库伦规范和洛仑兹规范具有相同的非齐次方程,说明了麦克斯韦方程组、洛仑兹规范与库伦规范都只是从不同的角度描述电磁场的运动规律,因此无论用何种方式描述电磁场,电磁场本身都没有改变。

论洛仑兹规范与库伦规范在静态场中的自恰性 洛仑兹规范与库伦规范在静态场中的 在静态场中 孙锴 (西安建筑科技大学 机电工程学院 ,陕西 西安 710055) 摘要: 摘要:本文通过麦克斯韦方程组引入电磁场规范 A, φ ,指出库伦规范和洛仑兹规范只是众 多电磁场规范 A, φ 中的两种较特殊的规范,最后推导出在静态场中库伦规范和洛仑兹规范 具有相同的非齐次方程, 说明了麦克斯韦方程组、 洛仑兹规范与库伦规范都只是从不同的角 度描述电磁场的运动规律,因此无论用何种方式描述电磁场,电磁场本身都没有改变。 关键词 关键词:洛仑兹规范;库伦规范;自恰性;矢量势;标量势 中图分类号: 中图分类号:O441.4; 0. 引言 . ( ) r ( ) r r 在电磁辐射中, 需要根据激发源来决定电磁场的性质, 而麦克斯韦方程组用电场强度 E r 和电磁感应强度 B 对电磁场进行描述,与激发源没有直接的联系,难以直接的描述电磁场, 因此,为了能够由激发源直接描述电磁场,引入了势函数的概念。 1. 电磁场的规范 A, φ 的引入 . 辐射电磁场中为了便于根据电荷电流计算场,常常使用标量势 φ 和矢量势 A 而非电场 ( ) r r r r 强度 E 和电磁感应强度 B 来描述电磁场。 真空中电磁场的麦克斯韦方程组的微分形式为: r ρ ??E = ε0 (1) r r ?B ?× E = ? ?t (2) r ??B = 0 r r r J ?E c 2? × B = + ε 0 ?t r ? ? (? × A) ≡ 0 (3) (4) 由矢量分析知旋度的散度为零,即: (5) 1 将(5)式代入(3)式,得 r r B = ?× A r (6) r 引入的矢势 A 只有横场部分具有确定的意义, 而其纵场部分可以任意取。 假定矢势 A 是 一个关于空间和时间的连续函数, ? 和 ? 可以交换微分次序。将(6)式代入(2)式,得 ?t r r r ? ?A ? × E = ? (? × A) = ?? × ?t ?t 整理,得 r ? r ?A ? ?=0 ?×?E + ? ?t ? ? ? ? × ?φ ≡ 0 (7) 由矢量分析知梯度的旋度度衡为零,即 (8) 比较(7)(8)式,引入标量势 φ ,得 、 r r ?A E=? ? ?φ ?t (9) 从(9)式可以看出,电场强度 E 不仅与标量势 φ 有关,还与矢量势 A 有关。矢量势 A 和标量势 φ 作为一组势函数,可以完备的描述一个辐射场,并且称 A, φ 为电磁场的规范。 2. 用电磁场的规范 A, φ 描述电磁场 将(9)式代入(1)式,得 r r r ( ) r ( ) r r ? ?A ? ρ ? ? ??? ? ?t ? ?φ ? = ε 0 ? ? 整理得 ? 2φ + r ? ρ ?? A = ? ε0 ?t (10) 将(9)式和(6)式同时代入(4)式,得 r r r J ? ?A c ?× ?× A = + (? ? ?φ ) ε 0 ?t ?t 2 ( ) 整理,得 r r 1 ?2 A r? r ? 1 ?φ ? A ? 2 2 = ? ? 0 J + ?? 2 + ? ? A? c ?t ? c ?t ? 2 (12) 2 从上面的推导我们可以看出, 麦克斯韦方程组中的四个方程分别独立推导出了四个标量 势 φ 和矢量势 A 的方程,他们分别是: r r r B = ?× A r r ?A E=? ? ?φ ?t ? 2φ + r ? ρ ?? A = ? ?t ε0 (6) (9) (10) r r 1 ?2 A r? r ? 1 ?φ ? A ? 2 2 = ? ? 0 J + ?? 2 + ? ? A? c ?t ? c ?t ? 2 (12) 势函数方程与麦克斯韦方程组的对应关系见表 1-1。 表 1-1. A, φ 规范的势函数方程的麦克斯韦方程组来源 麦克斯韦方程组 ( ) r (A,φ )规范的是函数方程 r r ρ ??E = ε0 ? 2φ + r ? ρ ?? A = ? ?t ε0 r r ?B ?× E = ? ?t r r ?A E=? ? ?φ ?t r ??B = 0 r r r J ?E c ?× B = + ε 0 ?t 2 2 r r B = ?× A r r 1 ?2 A r? r ? 1 ?φ ? A ? 2 2 = ? ? 0 J + ?? 2 + ? ? A? c ?t ? c ?t ? r r ?A 在静态场中,有 = 0 , E = ??φ ,因此 ?φ 是静电场的梯度,标量势 φ 是静电场的 ?t 电位函数。 3.静电场中洛仑兹规范过渡为库伦规范 .静电场中洛仑兹规范过渡为库伦规范 洛仑兹规范过渡 由亥姆霍兹定理知, 在无限空间中处处单值, 且导数连续有界而源分布在有限区域中的 矢量场 F 由其散度和旋度唯一确定。已知 B = ? × A ,若 ? ? A 也确定的话,矢量势 A 就 可以唯一确定。此时,规范 A, φ 可以唯一确定电磁场。 r r r r r ( ) r 3 由库伦规范: ? ? A = 0 ,得 r ? 2φ = ? ρ ε0 (13) r r 1 ?2 A r 1 ?φ ? 2 A ? 2 2 = ?? 0 J + ? 2 c ?t c ?t (14) r r ?A 在静态场中有 = 0 ,比较(9)式可知 E = ??φ 。因此 ?φ 是电场的梯度,标量势 φ ?t 是静电场的电位函数。 (13)式正是静电场的泊松方程,说明标量势 φ 是静电场的电位函数, 因此有 ?φ =0 ?t (14) 所以,在静态场中洛仑兹规范 ? ? A = ? 渡为库伦规范。 4.结束语 . r 1 ?φ = 0 。可见在静电场下,洛仑兹规范过 c 2 ?t 由以上推导可以看出麦克斯韦方程组,洛仑兹规范和库伦规范只是不同的角度描述同一 r r r 个电磁场。无论是选用麦克斯韦方程组的电场强度 E 和电磁感应强度 B ,还是选用洛仑兹 规范和库伦规范的 A, φ ,只要它们描述的是同一个电磁场,那么电场强度 E 、电磁感应强 度 B 、标量势 φ 和矢量势 A 就可以相互推导,其表现表现形式将是一致的。也就是说,在 同一个电磁场中标量势 φ 和矢量势 A 具有自恰性,表现的是同一个电磁场的不同表现形式。 ( ) r r r r 参考资料 [1] 郭硕鸿.电动力学[M].北京:人民教育出版社,1979. [2] 虞国寅,周国全.电动力学[M].武昌:武汉大学出版社,2008. Self-Consistency of Coulomb’s gauge and Lorentz’s gauge in the electromagnetostatic field Kai Sun (College of Mechanical and Electrical Engineering,Xi’an university of Architecture & Technology, Xi’an, Shannxi 710055, China) 4 Abstract: This paper proves that both Coulomb’s gauge and Loentz’s gauge are one of special forms of electromagnetic gauge (A,φ ) which discribes the same characteristics of motion of r Coulomb’s gauge and Loentz’s gauge electromagnetic field from different perspectives. Hence it derives the same form of nonhomogeneous equations from r electromagnetostatic field which proves that the selection of electromagnetic gauge A, φ has no effect to the electromagnetic field. Key words: Coulomb’s gauge; Loentz’s gauge; Self-Consistency; vector potential; scalar potential : 作者简介:孙锴,女, (1977-)西安建筑科技大学机电工程学院教师,主讲课程:电磁场与 电磁波 ( ) in the 5


库仑规范_理学_高等教育_教育专区

暂无评价|0人阅读|0次下载|举报文档
库仑规范_理学_高等教育_教育专区。这是讲的比较清楚的一个,且还包含其他规范,推荐

第五章 电磁波的辐射 Electromagnetic Wave Radiation 本章所研究的问题是电磁波的辐射。 本章所研究的问题是电磁波的辐射。方 法和稳恒场情况一样, 当考虑由电荷、 法和稳恒场情况一样 , 当考虑由电荷 、 电 流分布激发电磁场的问题时, 流分布激发电磁场的问题时 , 引入势的概 念来描述电磁场比较方便。 念来描述电磁场比较方便。 本章首先把势的概念推广到一般变化电 磁场情况,然后通过势来解辐射问题。 磁场情况,然后通过势来解辐射问题。 本章主要内容 电磁场的矢势和标势 推迟势 电偶极辐射 电磁波的干涉和衍射 电磁场的动量 §5. 1 电磁场的矢势和标势 Vector and Scalar Potential of Electromagnetic r 1、用势 A, ?描述电磁场 为简单起见,讨论真空中的电磁场: 为简单起见,讨论真空中的电磁场: r ??? D= ρ r ? r ??×E = ? ?B ? ? ?t ? r ??? B = 0 r ? r r ?D ??×H = j + ? ?t ? r r r r D=ε0E, B = ?0H . 针对磁场 引入 r ?? B = 0 r r B =?× A r 的物理意义可由下式看出: A的物理意义可由下式看出: r S 即在任一时刻, 沿任一闭合回路L的线积 即在任一时刻,矢量 A沿任一闭合回路 的线积 分等于该时刻通过以L为边线的曲面 的磁通量。 为边线的曲面S的磁通量 分等于该时刻通过以 为边线的曲面 的磁通量。 ∫ L r v r v A? dl = ∫∫ B? ds r 不能像静电场那样直接引入电势。 对于电场 E不能像静电场那样直接引入电势。由 Faraday电磁感应定律可得: 电磁感应定律可得: 电磁感应定律可得 r r r r ?B ? ?A ?×E = ? = ? (?× A) = ??× ?t ?t ?t r ? r ?A? ?×? E+ ? = 0 ? ?t ? ? ? r r ?A E + = ?? ? ?t 是标势不 是静电势 即 r r ?A E = ?? ? ? ?t r r ?B =?× A ? r ?r ?A ? ?E = ?? ? ?t ? 电磁场和势之间的关系如下 r ?A r r = 0时,且 E = ?? ? a) 当 A 与时间无关, 与时间无关,即 且 ?t 就直接归结为电势; 这时 ?就直接归结为电势; 注意: 注意: ? ? 混为一谈。 与电势 r (E = ?? ) 混为一谈。因为在非稳恒情 况下, 不再是保守力场,不存在势能的概念, 况下, E 不再是保守力场,不存在势能的概念, 这就是说现在的 ?,在数值上不等于把单位正电 荷从空间一点移到无穷远处电场力所做的功。 荷从空间一点移到无穷远处电场力所做的功。为 了区别于静电场的电势, 了区别于静电场的电势,把这里的 ? 称为标势 (Scalar potential)。 。 c) 在时变场中r 磁场和电场是相互作用着的 在时变场中, , 整体, 整体,必须把矢势 A 和标势 ? 作为一个整体来描 述电磁场。 述电磁场。 r r ?A b) 绝对不要把 E = ?? ? 中的标势 ? ? ?t r ? 种等价的方式, 种等价的方式,但由于 E 、B 和 A、 之间是微分 方程的关系, 方程的关系,所以它们之间的关系不是一一对应 r 的,这是因为矢势 A 可以加上一个任意标量函数 r 的梯度, 的梯度,结果不影响 B,而这个任意标量函数 r r r ?A ? 要发生影响, 的梯度在 E = ?? ? 中对 E 要发生影响,但 r ?t r ?A ? 将 E = ?? ? 中的?与此融合也作相应的 ?t r 变换, 保持不变。 变换,则仍可使 E 保持不变。 2、规范变换和规范不变性 r r r ? r 虽然 E 和 B,以及A 和 是描述电磁场的两 r r 述变换式: 述变换式: r ψ为任意的标量函数,即ψ =ψ(x,t),作下 设 为任意的标量函数, r r r ?A→A = A+? ′ ψ ? ? ?ψ ? ? →?′ =? ? ?t ? r ′ 很容易证明: 于是我们得到了一组新的 A . ?′ ,很容易证明: r r r ′ ψ ψ ?× A = ?×(A+? ) = ?× A+?×(? ) r r = ?× A= B r ′ ?A ?ψ ? r ?? ′ ? ? = ?? ? ? ( ) ? (A+? ) ψ ?t ?t ?t r ? ?A ? = ?? + (? ) ? ? (? ) ? ψ ψ ?t ?t ?t r ?A r = ?? ? = E ? ?t r r 由此可见, 描述同一电磁场。 由此可见,(A′ . ?′) 和 (A. ?) 描述同一电磁场。 a) 库仑规范(Coulomb gauge) 库仑规范(Coulomb r r 库仑规范条件为 ?? A= 0,即规定 A 是一个 r 有旋无源场(横场)。 )。这个规范的特点是 有旋无源场(横场)。这个规范的特点是 E的纵 ?具有无旋性 , 场部分完全由? 描述(即 ?? 具有无旋性),横 描述( r r ?A 描述( 具有无源性)。 )。由 场部分由 A描述(即 具有无源性)。由 ?t r r ?A ? E = ?? ? ?t r r ?A 可见, 可见,??? 项对应库仑场 E ,? ?t 对应着感应 库 r 场E 。 感 b) 洛仑兹规范(Lorentz gauge) 洛仑兹规范(Lorentz 是一个有旋有源场( 定 A是一个有旋有源场(即 A 包含横场和纵场两 部分) 部分),这个规范的特点是把势的基本方程化为 特别简单的对称形式。 特别简单的对称形式。 r 1 ?? 洛仑兹规范条件为 ?? A+ 2 = 0 ,即规 C ?t r r ?t 3、达朗贝尔(d’ Alembert)方程 达朗贝尔(d’ Alembert)方程 从Maxwell’s equations r ? ? D= ρ ? ? r ?r ?A ? ?E = ?? ? ?t ? 2 r r D=ε0E r 所满足的方程,得到: 出发推导矢势 A 和标势 ?所满足的方程,得到: r r 1 ?? r ? 2r 1 ? A ) = ??0 j ?? A? 2 2 ??(?? A+ 2 ? c ?t c ?t ? r ? 2 ?? ? + ?? A= ? ρ ? ?t ε0 ? a) 采用库仑规范 上述方程化为 r (?? A= 0) ρ ? 2 ?? ? = ?ε ? 0 r ? 2 r ??2 A? 1 ? A ? 1 ? (? ) = ?? r ? 0j 2 2 2 ? c ?t c ?t ? r 1 ?? b) 采用洛仑兹规范( ?? A+ 2 采用洛仑兹规范( = 0) c ?t 上述方程化为 ? 2 ρ 1 ?2? ?? ? ? 2 2 = ? c ?t ε0 ? r ? 2 r ? 2r 1 ? A ?? A? c2 ?t2 = ??0 j ? 这就是所谓达朗贝尔 达朗贝尔( 方程。 这就是所谓达朗贝尔( d’ Alembert )方程。 4、举例讨论 试求单色平面电磁波的势 Solution: Solution: 单色平面电磁波在没有电荷, 单色平面电磁波在没有电荷,电流分布的自 由空间中传播,因而势方程(达朗贝尔方程在 Lorentz规范条件下)变为波动方程: 规范条件下) 规范条件下 变为波动方程: 2 ? 2 1 ?? ?? ? ? 2 2 = 0 ? c ?t r ? 2 r 1?A ??2 A? =0 2 2 ? c ?t ? 其解的形式为: 其解的形式为: ? =?0e ? ? rr ? r r i(k?x?ωt) ?A= A e 0 ? r 1 ?? 由Lorentz规范条件 ?? A+ 规范条件 = 0,即得 2 c ?t r r rr i(k?x?ω ) t 1 ik ? A+ 2 (?iω?) = 0 c c2 r r ?= k?A ω 磁波,这是因为: 磁波,这是因为: r 这表明, 这表明,只要给定了 A ,就可以确定单色平面电 r r r r r r r B = ?× A= ik × A= ik ×(A + A ) 纵 横 r r r r = ik × A +ik × A 纵 横 r r 对于单色平面波而言) 0(对于单色平面波而言) = ik × A r 横 r r r ?A E = ?? ? ? = ?ik? + iω A ?t r c2 r r r = ik( k ? A + iω ) A = ?i c [ ω 2 ω r r r r 2 k(k ? A ? k A ) ] c2 r r r = ?i k ×(k × A ) ω c r r = ? k ×B ω r r ? = ?cn×B 2 r r r 具有横向分量, 如果取 A = A ,即只取 A具有横向分量,那么 横 有 r r r r k ? A= k ? A = 0 横 c2 r r ? = k ? A= 0 从而得到: 从而得到: 因此有: 因此有: ω r r r r r r ?B = ?× A= ik × A= ik × A 横 ? r r ?r r r ?A ?A = ? = iω = iω 横 ? A A ?E = ?? ? ?t ?t ? r r 其中: 其中: (k ? A = 0) 如果采用库仑规范条件,势方程在自由空间中变 如果采用库仑规范条件, 为 ??2? = 0 ? r ? 2 r 1 ?2 A 1 ? ? ?? A? 2 2 ? 2 ? = 0 c ?t c ?t ? 当全空间没有电荷分布时, 当全空间没有电荷分布时,库仑场的标势 ? = 0 , 则只有 r r 1?A 2 ? A? 2 2 = 0 c ?t 2 其解的形式为 rr r r i(k?x?ωt) A= A e 0 由库仑规范条件得到 r r r 即保证了 A 只有横向分量,即 A= A ,从而得到 只有横向分量, 横 r r r ?? A=ik ? A= 0 r r r r r r ?B = ?× A=ik × A= ik × A横 ? r r ?r r r ?A ?A ?E = ??? ? = ? = iωA= iωA横 ?t ?t ? r (?? A= 0) 通过例子可看到: 通过例子可看到: 库仑规范的优点是: 库仑规范的优点是:它的标势 ? 描述库仑作 r 求出, 用,可直接由电荷分布 ρ 求出,它的矢势 A只有 横向分量, 横向分量,恰好足够描述辐射电磁波的两种独立 偏振。 偏振。 r 洛仑兹规范的优点是: 洛仑兹规范的优点是:它的标势 ? 和矢势 A r 构成的势方程具有对称性。 构成的势方程具有对称性。它的矢势 A的纵向部 的选择还可以有任意性, 分和标势 ? 的选择还可以有任意性,即存在多余 的自由度。尽管如此, 的自由度。尽管如此,它在相对论中显示出协变 因此,本书以后都采用洛仑兹规范。 性。因此,本书以后都采用洛仑兹规范。 Class is Over! Thank you! Boys and girls!

Wednesday, May 4, 2016

Potential Theory in Classical Probability, Potential theory, harmonic functions, Markov processes, stochas- ... of classical analytic potential theory: Green kernels, Laplace and Poisson.

Potential Theory--ICPT 94: Proceedings of the International ...

https://books.google.com/books?isbn=3110146541
Josef Král - 1996 - ‎Mathematics
Proceedings of the International Conference on Potential Theory, Held in Kouty, Czech Republic, August 13-20, 1994 Josef Král ...

[PDF]Lectures on Potential Theory - Tata Institute of Fundamental Research

www.math.tifr.res.in/~publ/ln/tifr19.pdf
by M Brelot - ‎Cited by 231 - ‎Related articles
Lectures on Potential Theory. By. M. Brelot. Notes by. K. N. Gowrisankaran and. M. K. Venkatesha Murthy. Second edition, revised and enlarged with the help of ...
Missing: sgd

A little history of Submarine Groundwater Discharge (SGD ...

https://planetgeogblog.wordpress.com/.../a-little-history-of-submarine-gr...
Jan 28, 2014 - Although people already used SGD as an important water source for ... these deviations and created the basis for the Potential Theory and the ...

[PDF]Potential Theory in Classical Probability

www.ntu.edu.sg/.../greifswald_potenti...
Nanyang Technological University
by N Privault - ‎Cited by 5 - ‎Related articles
Key words: Potential theory, harmonic functions, Markov processes, stochas- ... of classical analytic potential theory: Green kernels, Laplace and Poisson.
Missing: sgd

Tuesday, May 3, 2016

A Data-Mining Approach to Travel Price Forecasting

https://hal.archives-ouvertes.fr/hal-00665041/document



https://hal.archives-ouvertes.fr/hal-00665041/document
A Data-Mining Approach to Travel Price Forecasting


http://www.cs.ru.nl/bachelorscripties/2014/Tim_Janssen___4150880___A_Linear_Quantile_Mixed_Regression_Model_for_Prediction_of_Airline_Ticket_Prices.pdf
A Linear Quantile Mixed Regression Model for Prediction of Airline Ticket Prices 


http://www.isi.edu/info-agents/papers/etzioni03-kdd.pdf


To Buy or Not to Buy: Mining Airfare Data to Minimize Ticket Purchase Price Oren Etzioni Dept. Computer Science University of Washington Seattle, Washington 98195 etzioni@cs.washington.edu Craig A. Knoblock Information Sciences Institute University of Southern California Marina del Rey, CA 90292 knoblock@isi.edu Rattapoom Tuchinda Dept. of Computer Science University of Southern California Los Angeles, CA 90089 pipet@isi.edu Alexander Yates Dept. Computer Science University of Washington Seattle, Washington 98195 ayates@cs.washington.edu ABSTRACT As product prices become increasingly available on the World Wide Web, consumers attempt to understand how corporations vary these prices over time. However, corporations change prices based on proprietary algorithms and hidden variables (e.g., the number of unsold seats on a flight). Is it possible to develop data mining techniques that will enable consumers to predict price changes under these conditions? This paper reports on a pilot study in the domain of airline ticket prices where we recorded over 12,000 price observations over a 41 day period. When trained on this data, Hamlet — our multi-strategy data mining algorithm — generated a predictive model that saved 341 simulated passengers $198,074 by advising them when to buy and when to postpone ticket purchases. Remarkably, a clairvoyant algorithm with complete knowledge of future prices could save at most $320,572 in our simulation, thus Hamlet’s savings were 61.8% of optimal. The algorithm’s savings of $198,074 represents an average savings of 23.8% for the 341 passengers for whom savings are possible. Overall, Hamlet saved 4.4% of the ticket price averaged over the entire set of 4,488 simulated passengers. Our pilot study suggests that mining of price data available over the web has the potential 



http://community.loyola.edu/dchillman/wp-content/uploads/sites/389/2016/03/Case-Assignment-Part-A.pdf 

airfairs 

Danielle Hillman Prof. Paul Lande EC 220 03 18 February 2016


http://www.graphpad.com/guides/prism/6/statistics/index.htm?stat_skewness_and_kurtosis.htm
http://cs229.stanford.edu/proj2012/Papadakis-PredictingAirfarePrices.pdf

Predicting Airfare Prices Manolis Papadakis Introduction Airlines implement dynamic pricing for their tickets, and base their pricing decisions on demand estimation models. The reason for such a complicated system is that each flight only has a set number of seats to sell, so airlines have to regulate demand. In the case where demand is expected to exceed capacity, the airline may increase prices, to decrease the rate at which seats fill. On the other hand, a seat that goes unsold represents a loss of revenue, and selling that seat for any price above the service cost for a single passenger would have been a more preferable scenario. The purpose of this project was to study how airline ticket prices change over time, extract the factors that influence these fluctuations, and describe how they're correlated (essentially guess the models that air carriers use to price their tickets). Then, using that information, build a system that can help consumers make purchasing decisions by predicting how air ticket prices will evolve in the future. We focused our efforts on coach-class fares. Related Work There has been some previous work on building prediction models for airfare prices using Machine Learning techniques [1] [2] [3]. The various research groups have focused on mostly different sets of features and trained their models on different kinds of flights. A major distinction among these projects is the specific trend they are trying to predict. Specifically, we can categorize projects into 2 approaches: studying the factors that influence the average price of a flight [2], or those that influence the price of a specific flight in the days leading up to departure [1] [3]. We will use this distinction in the definition of our model. There also exist commercial services, like Bing Travel (which actually evolved from the work in [1]), that perform this kind of prediction, but their models are not made public. 


Quantum Field Theory II An introduction to Feynman diagrams A course for MPAGS Dr Sam T Carr University of Birmingham carrst@theory.bham.ac.uk February 9, 2009


Linear response theory and the fluctuation-dissipation theorem There is a very strong link between correlation functions (fluctuations) and response functions to an external applied field (dissipation). For example, consider the expectation value %GS|ρˆ(!x,t)ρˆ(!x " ,t " )|GS#. (8.1) We have been thinking of this as a correlation function between two points and times. However, we can also think of it as applying some extra density at !x " ,t " and seeing how it affects the density at x,t. Time ordered correlation functions are what we have been learning to calculate using the diagrammatic technique. However, response functions are what experimentalists will measure - they will perturb the system in some way and see what it does in response. It is therefore very important for us to understand much better the relationship between response functions and correlation functions. In fact, the fluctuation-dissipation theory in it’s full glory can not be fully understood until we have looked at finite temperature Green functions - so we will revisit many of these concepts again later in Lecture 12. However, the importance of this topic means it is well worthwhile meeting it at least once before we leave the subject of zero temperature Green functions - which is the purpose of this lecture.


http://www.physics.miami.edu/~nearing/mathmethods/operators.pdf


Operators and Matrices You’ve been using operators for years even if you’ve never heard the term. Differentiation falls into this category; so does rotation; so does wheel-alignment. In the subject of quantum mechanics, familiar ideas such as energy and momentum will be represented by operators. You probably think that pressure is simply a scalar, but no. It’s an operator. 


https://www.math.stonybrook.edu/~scott/mat308.spr11/TraceDet.pdf

Linearization, Trace and Determinant MAT308 Spring 2011 Scott Sutherland, Stony Brook University In order to analyze a system of nonlinear equations, one important aspect is the idea of linearization near a fixed point (or equilibrium solution). More specifically, suppose that we have a system of the form dX~ dt = F~ (X~ ), and furthermore that for some initial condition X~ 0 we have F~ (X~ 0) = ~0. Then the constant solution X~ (t) = X~ 0 is an equilibrium. That is, the trajectory in the phase plane is just a single point. We want to analyze what happens to solutions that start near such a solution


https://www.cs.umn.edu/sites/cs.umn.edu/files/tech_reports/11-025.pdf
 A regression model for predicting optimal purchase timing for airline tickets William Groves and Maria Gini Department of Computer Science and Engineering, University of Minnesota {groves, gini}@cs.umn.edu Abstract Optimal timing for airline ticket purchasing from the consumer’s perspective is challenging principally because buyers have insufficient information for reasoning about future price movements. This paper presents a model for computing expected future prices and reasoning about the risk of price changes. The proposed model is used to predict the future expected minimum price of all available flights on specific routes and dates based on a corpus of historical price quotes. Also, we apply our model to predict prices of flights with specific desirable properties such as flights from a specific airline, non-stop only flights, or multi-segment flights. By comparing models with different target properties, buyers can determine the likely cost of their preferences. We present the expected costs of various preferences for two high-volume routes. Performance of the prediction models presented is achieved by including instances of time-delayed features, by imposing a class hierarchy among the raw features based on feature similarity, and by pruning the classes of features used in prediction based on in-situ performance. Our results show that purchase policy guidance using these models can lower the average cost of purchases in the 2 month period prior to a desired departure. The proposed method compares favorably with a deployed commercial web site providing similar purchase policy recommendations. 1 Introduction Adversarial risk in the airline ticket domain exists in two contexts: the adversarial relationship between buyers and sellers, and the competitive relationships that exist between multiple airlines providing the equivalent service. Buyers are often seeking the lowest price on their travel, while sellers are seeking to keep overall revenue as high as possible to maximize profit. Simultaneously, each seller must consider the price movements of its competitors to ensure that its prices remain sufficiently competitive to achieve sufficient (but not too high) demand. It is impossible to effectively address the problem of optimizing decision making from the buyer’s point of view without also considering both types of adversarial relationships. Sellers (airlines) make significant long term investments in fixed infrastructure (airports, repair facilities), hardware (planes), and route contracts. The specific details of these long term decisions are intended to roughly match expected demand but often do not match exactly. Dynamic setting of prices is the mechanism that airlines use to increase the matching between their individual supply and demand profile in order to attain the greatest revenue. A central challenge in the airline ticket purchasing domain is the information asymmetry that exists between buyers and sellers. Airlines have the ability to mine significant databases of historical sales data to develop models for expected future demand for each flight. Demand for a specific flight is likely to vary over time and will also vary based on the pricing strategy adopted by the airline. For buyers, it is generally best to buy far in advance of a flight’s departure because the prices tend to increase dramatically as the departure date approaches. But, airlines often violate this principle and adjust prices downward to increase sales. We make two novel contributions in this work: (1) a method of automated optimal feature set generation from the data that leverages a hierarchicalization of the available features to efficiently compute a feature set is proposed; (2) the addition of time-delayed observations to the feature vector fed to the machine learning 1 algorithm is performed. This allows anticipation of trends and more complex relationships between variables. For instance, we address pricing behaviors up to and beyond 60 days prior to departure, and we consider purchasing a flight on any airline for a specific date and city pair (previous work only considers the cost of a specific pair of flight numbers from two specific airlines). These ideas are then experimentally applied to prediction in the real-world airline ticket purchase domain. This paper presents models that also accommodate preferences of passengers about the number of stops in the itinerary or the specific airline to use. We believe this prediction task is both a more difficult task and generates models that are more useful for actual airline passengers.



http://liawww.epfl.ch/uploads/project_descriptions/project_344.pdf
Machine Learning for Predicting Flight Ticket Prices Supervisor: Igor Kulev November 9, 2015 1 Overview and Goal Airline companies have the freedom to change the flight ticket prices at any moment. Travellers can save money if they choose to buy a ticket when its price is the lowest. The problem is how to determine when is the best time to buy flight ticket for the desired destination and period. Airline companies use many different variables to determine the flight ticket prices: indicator whether the travel is during the holidays, the number of free seats in the plane etc. Some of the variables are observed, but some of them are hidden. The goal of this project is to use machine learning techniques to model the behavior of flight ticket prices over the time. In order to build and evaluate the model, you will use data that contains historical flight ticket prices for particular routes. 2 Project Steps • Parse and preprocess the flight ticket data that will be given to you • Apply few different machine learning models on the data set • Evaluate and compare the models 3 Required Skills The student should have knowledge in Machine learning. The student should know how to work with some programming language for machine learning (for example Matlab). References [1] O. Etzioni, R. Tuchinda, C. A. Knoblock, and A. Yates, “To buy or not to buy: mining airfare data to minimize ticket purchase price,” in Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2003, pp. 119– 128