Transcription
AIML428 Baseline is due this Friday 5pm, code for step 1 No demo for baseline Next Monday Kevin de Lange, Svetlana Koroteeva, Elijah Guarina, CallumMacaskill Today Last lecture on the first half of this course Some of my students’ projects Representation, Multi-view representation Learning the network architecture and meta parametersFeatures (Representation)masswidthheightcolor 21767.47.20.601907.15.60.752Text representation TF.IDF Word embedding Multiple sense embedding Sentence representation Higher level Document representation Multi-view representation1
Multi-view representationData RepresentationTime and CustomerProduct and CustomerTime and ProductDataViewsMultiple Views of DocumentsTermsTopicsAbdul Wahid, Xiaoying Gao, Peter Andreae:Multi-view clustering of web documents using multi-objective genetic algorithm. IEEECongress on Evolutionary Computation 2014: 2625-26322
Ontology based representation(Domain knowledge)An example: "Hyperlipidemia: The patient's Lipitor was increased to 80 mg q.d. A progress notein the patient's chart from her assisted living facility indicates that the patient hashad shortness of breath for one day."Detected Phraseshyperlipidaemia80shortness of breathExtracted Concepts[Disease or Syndrome][Finding][Quantitative Concept][Sign or Symptom][Clinical Attribute]Scientific NameSelectedhyperlipidemia80Dyspnea[Intellectual Product]one day[Temporal Concept] One day Mahdi Abdollahi, Xiaoying Gao, Yi Mei, Shameek Ghosh, Jinyan Li, Michael Narag:Substituting clinical features using synthetic medical phrases: Medical text data augmentationtechniques. Artif. Intell. Medicine 120: 102167 (2021)Term dependencyDependency Parse TreesDependency based word embeddings: context contains the words withdependency, may be far away in a sentence.Normal word embedding uses a sliding window of fixed size to get thecontext.Kosisochukwu Judith Madukwe, Xiaoying Gao, Bing Xue:Dependency-Based Embedding for Distinguishing Between Hate Speech and OffensiveLanguage. WI/IAT 2020: 860-868Emotion embedding NRC Word-Emotion Association Lexicon (EmoLex)[Mohammad2013] Every word in a document is transformed into a vector wi {0, 1}10containing the emotional information.3
Multi-view representation for fake news detectionFeature ExtractorsClassifierTuan Ha, Xiaoying Gao: Fake News Detection Using Multiple-View Text Representation. PRICAI (2)2021: 100-112Comparison of Features Projection of text representations (Red: fake news, Blue: real news)How to learn the multi-view representation?Tobias Tuan Ha and Xiaoying Gao Evolving Multi-viewAutoencoders for Text Classification, WI-IAT21,4
Auto-encoders A class of neural networks, which are trained to attempt toapproximately copy their input x to their output x̂.With the simplest Autoencoder, the learning process isdescribed as minimising the loss functionL(x, g(f(x)))where L is a loss function such as Mean Squared Error(MSE)Autoencoder to learn multi-view representationLoss function for learning simultaneously In order to train the Autoencoders and the classifier, weintroduce an objective function containing two types oflosses: the Mean Squared Errors (MSE) as the reconstructionerrors of the Autoencoders, and the Cross Entropy as the loss of the classifier.5
Leaning the network architecture Learning methods Evolutionary Computation Genetic algorithms Genetic programming Character level model Word level model BERT based modelTrevor Londt, Xiaoying Gao, Peter Andreae:Evolving Character-Level DenseNet Architectures UsingGenetic Programming. EvoApplications 2021: 665-680Hayden Andersen, Sean Stevenson, Tuan Ha, Xiaoying Gao, Bing Xue:Evolving Neural Networks for Text Classification using Genetic Algorithm-based Approaches. CEC2021: 1241-1248Hayden Andersen, Xiaoying Gao, Bing Xue, Mengjie Zhang:Evolving network structures for text classification using genetic algorithms. GECCOCompanion 2020: 109-110Kosisochukwu Judith Madukwe, Xiaoying Gao, Bing Xue:A GA-Based Approach to Fine-Tuning BERT for Hate Speech Detection. SSCI 2020: 2821-2828CLS token, Layers, which layer?Discussion on research6
,0/ µ%dvholqh lv gxh wklv )ulgd\ sp frgh iru vwhs µ1r ghpr iru edvholqh µ1h[w 0rqgd\ µ.hylq gh /dqjh 6yhwodqd .rurwhhyd (olmdk *xdulqd &dooxp
