Signal and Data Processing

Files are the most important sources of information presenting in various formats such as texts, audio, video, images, web pages, etc. …; (in-depth) analysis of files for the purpose of recognition and investigating their unique properties (or characteristics) is one of the most significant issues in the field of personal security safety, information security, file-type identification, codes structuration analysis etc…. Statistical analytic methodology of working on the binary files contents based on the n-gram model has been opted for in the present paper in order to full investigate all different aspects of a file’s range of characteristics. Moreover, to reduce down the calculations volume and the n-gram model peculiar to the needed amount of memory, use has been made of word clustering. Later on analysis has been conducted on both files’ contents in two states of “blocking” and “full”: it is to be noted that in the “full” case such characteristics as Chi-square, Auto-correlation, Weighted term frequency-Inverse document frequency (TF-IDF), Fractal dimension etc … have been brought under comprehensive study; while in the “blocking” case, other properties like the entropy rate, the distance, etc … have been delved into. The gained results indicate that the extracted characteristics in the first method could well easily reflect the unique properties belonging to jpg, mp3, swf and html files; and in the second method, are able to clearly well reflect doc, html and pdf files properties.

The onset of social media venues, online news media, and digital content allowed a vast volume of text and time series data to be generated which plays significant role in investors' decision-making and financial market volatility. Data extracted from these platforms provide information on public sentiments, immediate reactions to news, and informal analyses, which, if processed appropriately, can be very useful indicators in forecasting financial market trends. Billions of dollars are invested and lost, depending on correct forecasting. However, advances in deep learning, especially in large language models (LLMs) and novel time series analysis algorithms, have opened new windows to processing and analyzing this complex data. The advanced language models identify hidden patterns and nonlinear dependencies, always taking into account the context and semantic details of the text between news, market sentiments, and price fluctuations, as well as utilizing them via intelligent market analysis systems. This review analyzes the existing research trends on the relationship of text data available on websites and social networks with the behavior of financial markets, having reviewed more than 200 scientific papers published between 2006 and 2024 in a systematic manner. This study focuses on identifying advanced methods within text representation, sentiment analysis, predictive modeling, and language model applications for analyzing real-time and unstructured data. More than one information source has to be taken into consideration: (Twitter, news agencies, blogs, and specialized forums) from a perspective of credibility, data structure, and influence-on market decisions. Given the complexity of financial markets, such as stocks and forex, there is an ever-increasing demand for hybrid models capable of carrying out analyses across time-series and text data simultaneously. This paper aims to analyze the current research accomplishments, identify gaps in the research, and ultimately put forward future directions for the fields of text mining, AI, and deep learning. These directions can open up the path for the next generation of real-time and adaptive recommender, predictor, and correlation analyzer systems in the financial markets.