Analysis and prediction in sparse and high dimensional text data: The case of Dow Jones stock market

Basit Öğe Kaydı
dc.authorid0000-0001-6657-9738
dc.contributor.authorSert, Onur Can
dc.contributor.authorŞahin, Salih Doruk
dc.contributor.authorÖzyer, Tansel
dc.contributor.authorAlhajj, Reda
dc.date.accessioned2020-01-22T06:37:41Z
dc.date.available2020-01-22T06:37:41Z
dc.date.issued2020
dc.departmentİstanbul Medipol Üniversitesi, Mühendislik ve Doğa Bilimleri Fakültesi, Bilgisayar Mühendisliği Bölümü
dc.description.abstractIn this research, we proposed a text analysis system to predict stock market movements using news and social media data. It is a scalable prediction system for sparse and high dimensional feature sets. Using the developed system, we collected 12,560 articles from New York Times covering one year time period, and 2,854,333 tweets from Twitter covering 4 months time period. We analysed the collected data using entity extraction, sentiment analysis and topic modelling techniques. We applied our feature set creation and elastic net regression based training method. The analyses have been used to train different prediction models. Using these trained prediction models, we predicted stock market movements for Dow Jones Index and showed that the proposed method can make promising predictions. In different sets of experiments, highly accurate (up to 70.90% accuracy) predictions are made by the proposed approach. These predicted values also correlated (up to 0.2315 correlation coefficient value) with real Dow Jones Index values. Further, we report performance comparison results for various prediction models that we trained with different set of features to analyse the importance of time interval and feature space size. Our test results show that it is possible to make reasonable stock movement prediction by integrating news and related social media data, analysing them using named entity extraction, sentiment analysis and topic modelling techniques together with prediction models which use features that are created from these analysis results.
dc.identifier.citationSert, O. C., Şahin S. D., Özyer, T. ve Alhajj, R. (2020). Analysis and prediction in sparse and high dimensional text data: The case of Dow Jones stock market. Physica A: Statistical Mechanics and its Applications, 545. https://doi.org/10.1016/j.physa.2019.123752.
dc.identifier.doi10.1016/j.physa.2019.123752
dc.identifier.issn3784371
dc.identifier.scopusqualityQ2
dc.identifier.urihttps://doi.org/10.1016/j.physa.2019.123752
dc.identifier.urihttps://hdl.handle.net/20.500.12511/4911
dc.identifier.volume545
dc.identifier.wosqualityQ2
dc.indekslendigikaynakWeb of Science
dc.indekslendigikaynakScopus
dc.language.isoen
dc.publisherElsevier B.V.
dc.relation.ispartofPhysica A: Statistical Mechanics and its Applicationsen_US
dc.relation.publicationcategoryMakale - Uluslararası Hakemli Dergi - Kurum Öğretim Elemanı
dc.rightsinfo:eu-repo/semantics/embargoedAccess
dc.subjectNamed Entity Recognition
dc.subjectTopic Modelling
dc.subjectSentiment Analysis
dc.subjectSocial Network Analysis
dc.subjectStock Market Movement Prediction
dc.subjectMsaene
dc.titleAnalysis and prediction in sparse and high dimensional text data: The case of Dow Jones stock market
dc.typeArticle

Dosyalar

Orijinal paket
Listeleniyor 1 - 1 / 1
Küçük Resim Yok
İsim:
Alhajj, Reda-2020.pdf
Boyut:
3.4 MB
Biçim:
Adobe Portable Document Format
Açıklama:
Tam Metin / Full Text
İndir

Koleksiyon

Scopus İndeksli Yayınlar Koleksiyonu
Makale Koleksiyonu
WoS İndeksli Yayınlar Koleksiyonu