Back from corn field !

Reading reading and reading. Felt like a bear picking the corn in the field, kept throwing the previous corn he got, while searching for the biggest corn. Now it's time to get back, and happy with the current corn.

11:40am - 3.40pm: Producing the Rouge sccore for version w/ and w/o compression. Change the code to take the input as *_orig.txt; Now waiting for the permission to run the Rouge.

4:00pm -6:30pm : Did the content and linguist5 comparison . Try to find what are the factors that affect coherence score in DUC 06. How much difference when question is added .

8:00pm - 9:30pm: Analysis continues. Write a doc version analysis.

10:50pm - 11:50pm: DUC paper updates (Planend)

Weekly updates --- Linguistics or newpaper reader...

Read extensively of DUC 05 and DUC06 data recently, and try to find good examples for the hypothesis of topic clustering. To my disappointment, only found one so far. Some topics, the clustering method failed to group them into the right category.

Based on the analysis of DUC 06 data, many of those who have higher Structure and Coherence score have only one question. So topic clustering here doesn't work at all.

Those topics that system performs well are due to several reasons:
1. Most sentences are the first sentence in the original doc;
2. Neighbouring sentences from the same doc are grouped together.
3. Lexical cohesion.

Another discovery is that Structure and coherence score have some correlations with the Content responsiveness. This means if an article has a good content selection, chances of its being coherent are high. If it is poor content selected, it will most likely be rated as incoherent. So coherence here really counts on the content.

One more possible try is to use the human summary , and check if the topic clustering help to group or not. Since the human summary answer questions by topic.

Got pretty frustrated. :( Sentence ordering is too subjective, so part of the research is to be a newspaper reader?

Hypothesis and plans ahead

11:20- 12:20 Meeting with Anoop.

Scenerial: Multiple question based summarization.

Finding the hypothesis:

There are two places that multiple questions might affect the ordering stratege and the quality of the final summary.

1. If the shallow semantic-related(concept ID overlap) sentences answer different questions, then they should be put in the different cluster group in the summary.

2. Use questions to help picking the sentences for ordering. Since the input to the Editor is the twice the size of the final summary. Based on the importance score of the sentences from Extractor, it is possible that all the sentences that answer a specific questions ranked higher while ommiting the other questions. By using question-directed clustering, it garantees that every question is answered in a good portion.


Planned work:
Check out the DUC05 data, based on PE results and human summary. Identify the relations between sentences answering different questions.

Work plan:
1. Identify summary/topic that has higher Responsiveness and Rouge Score to garantee the relateness of sentence.
2. Manually create an order of the summaries. Identify the lexical cohesions.
3. Discover the patterns of human summary.

Dada analysis Plan: DUC05 data
1. Analyze the human summary and see if they structure summaries by questions.
2. Get all the SCU marked sentences and divide into question group, according to human summaries. Then create new ordered summaries (human work involves).

Question for Gabor: Did extractor extracted sentences question by question or consider them as a whole?
Answer: Extractor treated the questions as a bag of WordNet concepts. It was unmoved by the number of questions asked.

Get DUC2006 data topic analysis done:
Discover: Squash did equally well in responsiveness in Content and Overall. Some topics do improve in overall based on the content info. Will check out by reading summaries tomorrow.

Chomsky Hierachy

Went to the TAG study group for 1.5 hours. It is very interesting. Reviewed again the Chomsky hierachy. Learned more about String/Tree Grammar. Felt very happy when I am learning.
http://www.cs.sfu.ca/~anoop/courses/ReadingGroup-Summer-2006/index.html

Pull up the document for the preparation with Anoop's meeting tomorrow.

Found some new paper about question and summarization.
Need to investigate more.

Need to learn to write up things. Need to put things down from reading and thinking, so as to make real noticable progress.

May 8th --- Meetings, Statistic analysis and more

DUC Team Meeting:
Clarify the DUC2006 paper written task: Editor + Results Section

Fred Meeting:
Keep on track. Focus on Query. Propose the evaluation plan. Keep on review writing.

Gabor Meeting:
Write a paragraph per section discribe the overview and problem that's going to address.

Learned severl Excel tricks from Gabor>>>

Test the correlation of two sets of data. R Squared value.
DrawChart -> XY Scatter -> Add Trendline -> Type: Linear / Options: Display R-squared value.

Tool -> DataAnalysis -> Descriptive Statistics
The Descriptive Statistics analysis tool generates a report of univariate statistics for data in the input range, providing information about the central tendency and variability of your data.

Weekly status report 2

Work has done:
1. Finish the literature review on the ordering methodology and evaluation, get a simple chart summary as the outline of write-up in the thesis.
2. Didn't discover more literature from the perspective of query-based summarization.
3. Analyzing data. Try to read summaries and one topic original docs to identify the chronological info. and the usefulness of query clustering.
4. Finish a eval result draft report on DUC2006 data.

Work to be done:
Keep in mind "Query is the focus" !
1. Discuss the eval experiment set up and form the initial agreement.
2. Discuss the methods and find our special hypothesis, re-design the algorithm
3. DUC06 results analysis continues, preparing for May25th paper.
4. Continue the literature review and introduction parts write-up in thesis.
5. Daily blog.

Weekly Status Report

Status Report:

Work has been done:
1. Set up the Latex thesis working enviroment. Putting up what have written before into the thesis sketch. Using JabRef to manage all the reference
2. Found out most of the literatures that to be read regarding ordering and evaluation. Finish 1.5 round reading. Wrote 5 reviews.
3. Look into the statistic correlation evaluation metrics. ANOVA, Spearman and kendall metrics.
4. Analysing DUC06 results ( 1 day )

Work to be done:
1. Have a write-up on the statistic correlation evaluation metrics.(Next Week)
2. Finish the write-up for literature review on the ordering methodology and evaluation, thesis version ready as well.
3. Search more literature from the perspective of query-based summarization. Try to find the data difference.
4. Evaluation on DUC2006 data.
5. Have a daily blog on the working progress.