AI Image Generators

Explore the best AI Image Generators — independent reviews, comparisons, pricing and step-by-step how-to guides, curated by Aizhi.

17LIVE

17LIVE is an international entertainment platform. As of 2024, 17LIVE is the #3 live broadcasting platform globally, formed by its flagship live stream app 17LIVE (LIVIT in English markets), MEME Live and live stream e-commerce platforms HandsUP and OrderPally. == History == 17LIVE was first founded in Taiwan in 2015 by Jeffery Huang. The company has maintained its leading position since its entry into the Japan market in 2017, becoming the biggest platform for live entertainment in Japan, Taiwan, Hong Kong, and other countries. In 2017, 17 closed out US$33M in series B round to merge with dating software Paktor, with Joseph Phua (Co-founder of Paktor) taking over the leadership of 17LIVE as CEO and Co-founder, as well as to enter the Japan and Hong Kong market. Within one year, 17 Media became the #1 market leader in Japan. In 2018, the company raised $25M in series C round as it got ready for US IPO, which failed to materialize. 17LIVE had an unsuccessful US IPO attempt in 2018. Since then, the company reformed and transformed the business. Some key initiatives include the hiring of current CEO Hirofumi Ono, spin-off of Paktor (dating software business unit), full buy-out of founder Jeffery Huang, acquisition of MEME and HandsUp, and more. Despite the failed IPO attempt, the company continued to push for international expansion, including creating ‘LIVIT’ for the English-speaking markets to enter US, India, and North Africa. In 2019, 17's flagship live streaming app reached 10M downloads in Japan, and the business continues to push for both organic and inorganic expansion. Some key M&A highlights in the year include the acquisition of MEME Live in Southeast Asia, as well as HandsUp, a live e-commerce platform. In 2020, M17 closed out $26.5M in Series D round to continue organic growth in Japan, US and Middle East. In the same year, the company also sold its dating app business, Parktor, to rationalise M17 into a live-stream pure play business, followed by the appointment of its current Chairman, Joseph Phua, and previous Global CEO, Hirofumi Ono. With the buy-out and departure of founder Jeff Huang, the parent holding company M17 Entertainment Limited was officially renamed as 17 LIVE Group. An estimated 60 million users registered in 154 countries and territories in April 2022. In 2022, September, 17LIVE announced Group CEO Hirofumi Ono steps down. Alex Lien takes over the leadership as new Group COO; Jing Shen Ng appointed Group CTO. In 2023, March, 17LIVE announced Alex Lien promoted to Global CEO. Kenta Masuda appointed as Global CFO. === Collaboration with Ayumi Hamasaki === To celebrate its 4th anniversary, 17LIVE collaborated with Japanese singer-songwriter Ayumi Hamasaki, who led the 17LIVE 4th Anniversary meets Ayumi Hamasaki series starting October 18, 2021. Along with composer and arranger Yuta Nakano, Hamasaki judged auditioning artists competing for the chance to work with her and her production team for a debut single. The series was streamed live on the 17LIVE website, the final airing on November 11. The eventual winner was named as Yoshitaka_song. When asked why she collaborated with 17LIVE as a producer, Hamasaki commented: "Although the world has become like this (during COVID-19), I believe that the art of entertainment can give people dreams, hope, courage, and strength. I hope that kind of light will continue to shine through the entertainment industry." == Features == On 17LIVE, artists (LIVERs) are able to broadcast live, and post photos and videos from their album. The app has been designed for LIVERs to simply open the App, and start sharing contents without the need to edit or professionally curate their videos. The platform cultivates LIVERs, supports them with a local content management team, and provides artists with various functions, such as real time chatting, gifting, fan clubs, interactive competition and events. Today, 17LIVE has 46 thousands contracted artists and more than 2.3 million MAU, who spend 44 minutes on the platform every day. 17LIVE continues to advocate content-driven philosophy and delivers diverse topics, from politics and music to entertainment, to broaden its audience groups. 17LIVE also hosts offline flash events and concerts to attract new users and support LIVERs better connect with their fans. == Operation == 17LIVE has over 700 employees globally. The app provides few monetization models for LIVERs on the platform, including: Gifting: user / fans buy virtual gifts on the app to send to their favored LIVERs. Subscription: monthly subscription fan club service for access to exclusive content Pay-per-view: ticket service for online streaming concerts E-commerce: live e-commerce platform In the past, 17LIVE has encountered some regulatory headwinds with reported incidents of inappropriate livestream content on the platform. The incidents were direct results of the lack of oversight and supervision capability in place in the business at the time. Over the years, 17LIVE claims to have put in tremendous manpower and effort into improving, monitoring and maintaining control over both the live stream content and the KYC procedures and systems.
Read more →
Social news website

A social news website is a website that features user-posted stories. Such stories are ranked based on popularity, as voted on by other users of the site or by website administrators. Users typically comment online on the news posts and these comments may also be ranked in popularity. Since their emergence with the birth of Web 2.0, social news sites have been used to link many types of information, including news, humor, support, and discussion. All such websites allow the users to submit content and each site differs in how the content is moderated. On the Slashdot and Fark websites, administrators decide which articles are selected for the front page. On Reddit and Digg, the articles that get the most votes from the community of users will make it to the front page. Many social news websites also feature an online comment system, where users discuss the issues raised in an article. Some of these sites have also applied their voting system to the comments, so that the most popular comments are displayed first. Some social news websites also have a social networking service, in that users can set up a user profile and follow other users' online activity on the website. Like many other Web 2.0 tools, social news websites use the collective intelligence of all of the users to operate. Social news websites also "impl[y] the technical, economic, legal, and human enhancement of a universally distributed intelligence that will unleash a positive dynamic of recognition and skills mobilization". Social news websites help participants to share a collective vision and awareness of how their actions are integrated with those of other individuals. Social news websites provide a new and innovative way to participate in a community that is constantly being flooded with new information. These social news websites "include opportunities for peer-to-peer learning, a changed attitude toward intellectual property, the diversification of cultural expression, the development of skills valued in the modern workplace, and a more empowered conception of citizenship". These websites can help to shape and reshape democratic opinions and perspectives. Social news sites may mitigate the gatekeeping of mainstream news sources and allow the public to decide what counts as "news", which may facilitate a more participatory culture. Social news sites may also support democratic participation by allowing users from across geographic and national boundaries to access the same information, respond to fellow users' views and beliefs, and create a virtual sphere for users to contribute within. == Websites == === Active === ==== Fark ==== Fark, which started in 1997, features news on any topic. On Fark, users can submit articles to the administrators of the site. Each day, these administrators pick out 50 articles to display on the front page. ==== Slashdot ==== Slashdot, started in 1997, was one of the first social news websites. It focuses mainly on science and technology-related news. Users can submit stories and the editors pick out the best stories each day for the front page. Users can then post comments on the stories. The influx of web traffic that resulted from Slashdot linking to external websites led to the effect being called the Slashdot effect ==== Digg ==== Digg, started in December 2004, introduced the voting system. This system allows users to "digg" or "bury" articles. "Digging" is the equivalent of voting positively, so that popular articles are displayed first. "Burying" does not lower an article's score. However, if an article is buried enough times, it will be automatically deleted from the site. Digg offers a social networking service, as members can follow other members and build personal profiles with information about their interests. ==== Reddit ==== Reddit, started in June 2005, is a social news website where users can submit articles and comments and vote on these submissions. The submissions are organized into categories called "subreddits". Unlike Digg, with Reddit, users can directly affect an article's score. An "upvote" will increase the score and a "downvote" will decrease it. Articles with the highest scores are displayed on the front page. There is also a page for "controversial" articles, that have an almost equal number of upvotes and downvotes. Free speech debates have arisen due to the shutting down of obscene or potentially illegal "subreddits" (including /r/jailbait, a collection of sexually suggestive underage pictures.) Reddit introduced a system of user-created communities called "subreddits", which are essentially categories for a specific type of news. Comments on the featured posts are shown in a hierarchical fashion also based on votes. Users have the ability to earn "karma" for their participation and time on the website. ==== Hacker News ==== Hacker News, started in February 2007, is a social news site focusing on computer science and entrepreneurship, created by Paul Graham and run by his startup incubator, Y Combinator. === Defunct === ==== Newsvine ==== Newsvine, started in March 2006, was a social news website mostly focused on politics, both international and domestic. The Newsvine home page allowed users to customize "seeds" and story feeds. Users received articles via "The Wire" from sources including The Associated Press or The Huffington Post, and from "The Vine" a stream of content from other Newsvine users. The "Top of the Vine" displayed the most voted and commented on articles of the day, week, month, or year. Additionally, Newsvine allowed members to create their own "Customizable Column", which could highlight a user's content posted, recent comments, and information about the specific Newsvine member. ==== feedalizr ==== feedalizr was a cross-platform, desktop social media aggregator built using Adobe Integrated Runtime that consolidates the updates from social media and social networking websites. Users can then use this application to update those sites from their desktop and view a consolidated stream of information. ==== Voat ==== Voat, launched in April 2014 and discontinued in December of 2020, was also a social news website and is very similar to Reddit visually and functionally. The site's userbase included a large number of alt right users, many of whom migrated to Voat after being banned on Reddit. ==== Prismatic ==== Prismatic combined machine learning, user experience design, and interaction design to create a new way to discover, consume, and share media. Prismatic software used social network aggregation and machine learning algorithms to filter the content that aligns with the interests of a specific user. Prismatic integrated with Facebook, Twitter, and Pocket to gather information about user's interests and suggest the most relevant stories to read. ==== Artifact ==== Artifact was an iOS and Android app that used machine learning to personalize news recommendations to readers, and also had social features such as liking articles, commenting, and reputation scores for users.
Read more →
Content format

A content format is an encoded format for converting a specific type of data to displayable information. Content formats are used in recording and transmission to prepare data for observation or interpretation. This includes both analog and digitized content. Content formats may be recorded and read by either natural or manufactured tools and mechanisms. In addition to converting data to information, a content format may include the encryption and/or scrambling of that information. Multiple content formats may be contained within a single section of a storage medium (e.g. track, disk sector, computer file, document, page, column) or transmitted via a single channel (e.g. wire, carrier wave) of a transmission medium. With multimedia, multiple tracks containing multiple content formats are presented simultaneously. Content formats may either be recorded in secondary signal processing methods such as a software container format (e.g. digital audio, digital video) or recorded in the primary format (e.g. spectrogram, pictogram). Observable data is often known as raw data, or raw content. A primary raw content format may be directly observable (e.g. image, sound, motion, smell, sensation) or physical data which only requires hardware to display it, such as a phonographic needle and diaphragm or a projector lamp and magnifying glass. The following are examples of some common content formats and content format categories (covering: sensory experience, model, and language used for encoding information):
Read more →
Data definition specification

In computing, a data definition specification (DDS) is a guideline to ensure comprehensive and consistent data definition. It represents the attributes required to quantify data definition. A comprehensive data definition specification encompasses enterprise data, the hierarchy of data management, prescribed guidance enforcement and criteria to determine compliance. == Overview == A data definition specification may be developed for any organization or specialized field, improving the quality of its products through consistency and transparency. It eliminates redundancy (since all contributing areas are referencing the same specification) and provides standardization and degrees of compliance, making it easier and more efficient to create, modify, verify, analyze and share information across the enterprise. To understand how a data definition specification works in an enterprise, we must look at the elements of a DDS. Writing data definitions, defining business terms (or rules) in the context of a particular environment, provides structure for an organization's data architecture. In developing these definitions, the words used must be traceable to clearly defined data. A data definition specification may be used in the following activities: Business intelligence Business process modeling Business rules management Data analysis and modeling Information architecture Metadata modeling Data mastering Report generation == Criteria == A data definition specification requires data definitions to be: Atomic – singular, describing only one concept. Commonly used and ambiguous terms should be defined. While a term refers to one concept, several words may be used in a term: File – A concept identifiable with one word File extension – A concept identifiable with more than one word Traceable – Mapped to a specific data element. In business, a term may be traced to an entity (for example, a customer) or an attribute (such as a customer's name). A term may be a value in a data set (such as gender), or designate the data set itself. Traceability indicates relationships in the data hierarchy. Consistent - Used in a standard syntax; if used in a specific context, the context is noted Accurate - Precise, correct and unambiguous, stating what the term is and is not Clear - Readily understood by the reader Complete - With the term, its description and contextual references Concise - To avoid circular references == Applications == === Enterprise data === A data definition specification was produced by the Open Mobile Alliance to document charging data. The document, the centralized catalog of data elements defined for interfaces, specifies the mapping of these data elements to protocol fields in the interfaces. Created for the exchange of financial data, Market Data Definition Language (MDDL) is an XML specification designed to enable the interchange of information necessary to account, to analyze, and to trade financial instruments of the world's markets. It defines an XML-based interchange format and common data dictionary on the fields needed to describe: (1) financial instruments, (2) corporate events affecting value and tradability, and (3) market-related, economic and industrial indicators. The principal function of MDDL is to allow entities to exchange market data by standardizing formats and definitions. MDDL provides a common format for market data so that it can be efficiently passed from one processing system to another and provides a common understanding of market data content by standardizing terminology and by normalizing the relationships of various data elements to one another ... From the user perspective, the goal of MDDL is to enable users to integrate data from multiple sources by standardizing both the input feeds used for data warehousing (i.e., define what's being provided by vendors) and the output methods by which client applications request the data (i.e., ensure compatibility on how to get data in and out of applications)." === Clinical submissions === The Clinical Data Interchange Standards Consortium, a global, multidisciplinary, non-profit organization, has established standards to support the acquisition, exchange, submission and archiving of clinical research data and metadata. CDISC standards are vendor-neutral, platform-independent and freely available from the CDISC website. The Case Report Tabulation Data Definition Specification (define.xml) draft version 2.0, the oldest data definition specification, is part of the evolution from the 1999 FDA electronic submission (eSub) guidance and electronic Common Technical Document (eCTD) documents specifying that a document describing the content and structure of included data be included in a submission. Define.xml was developed to automate the review process by generating a machine-readable data-definition document. Define.xml has standardized submissions to the Food and Drug Administration, reducing review times from over two years to several months. === Archival data === A data definition specification is the foundation of metadata for scientific data archiving. The Metadata Encoding and Transmission Standard (METS) uses one principle of a DDS: consistent use of key terms to catalog digital objects for global use. The METS schema is a flexible mechanism for encoding descriptive, administrative and structural metadata for a digital library object and expressing complex links between metadata, and can provide a useful standard for the exchange of digital-library objects between repositories. A similar effort is underway to preserve complex data associated with video-game archiving. Preserving Virtual Worlds attempted to address archival-format deficiencies, citing the lack of suitable documentation for interactive fiction and games at the bit level: specifically, the absence of "representation information" needed to map raw bits into higher-level data constructs. Preserving Virtual Worlds 2 is a research project expanding on initial efforts in this field.
Read more →
Connection string

In computing, a connection string is a string that specifies information about a data source and the means of connecting to it. It is passed in code to an underlying driver or provider in order to initiate the connection. Whilst commonly used for a database connection, the data source could also be a spreadsheet or text file. The connection string may include attributes such as the name of the driver, server and database, as well as security information such as user name and password. == Examples == This example shows a PostgreSQL connection string for connecting to wikipedia.com with SSL and a connection timeout of 180 seconds: DRIVER={PostgreSQL Unicode};SERVER=www.wikipedia.com;SSL=true;SSLMode=require;DATABASE=wiki;UID=wikiuser;Connect Timeout=180;PWD=ashiknoor Users of Oracle databases can specify connection strings: on the command line (as in: sqlplus scott/tiger@connection_string ) via environment variables ($TWO_TASK in Unix-like environments; %TWO_TASK% in Microsoft Windows environments) in local configuration files (such as the default $ORACLE_HOME/network/admin.tnsnames.ora) in LDAP-capable directory services
Read more →
Social media use in the financial services sector

Social media in the financial services sector refers to the use of social media by the financial services sector to promote and distribute financial services. Social media is used in various aspects of the financial industry including customer service, marketing, and product development. It has enabled financial institutions to extend their reach through direct and real-time communication with customers, fostering more personal connections. It also allows individuals to talk to other individuals creating lending and trading via social groups as well as developing new financial services by fintech startup companies. In terms of marketing, social media is utilized by both traditional financial companies as well as disruptive fintech companies such as peer-to-peer lending (P2P) companies. The financial industry has used information technology since its inception in the 1960s and social media fits in with this ongoing development. Larger, traditional financial firms have integrated social media into their marketing strategies. Companies in the financial sector are subject to strict regulations that include how they use social media. In the United States, the Financial Industry Regulatory Authority (FINRA) is a key regulator that sets rules how financial firms can interact with consumers. This includes ensuring that social media posts follow financial advertising rules, such as being fair and balanced and not providing misleading information, and that financial advice is not provided by unqualified personnel, such as influencers. == History == In 2003, at the beginning of social media development, MySpace was founded as a "social networking service." It allowed people to create a profile, connect with other people, and post videos, pictures, and songs. As MySpace grew in popularity, it attracted interest from companies wishing to promote their brands on the social platform. They were joined by Facebook and in 2010 by Instagram. Financial service firms were initially slow to adapt to promotion via social media but soon joined other big firms after they saw the success other industries had in engaging with younger people. == Uses == === Branding === While companies are able to connect with more people remotely through providing online financial services, their branding strategy has shifted from customized to standardized. Prior to the outbreak of technology, most banks used customized branding where they targeted only customers in their regions. Businesses can now use technology to operate beyond their geographic location and maintain a consistent image across multiple countries with standardized branding. By being able to extend a consistent brand reputation across a wider geographic location, financial services companies can take advantage of economies of scale in advertising cost, lower administrative complexity, lower entry into new markets, and improved cross-border learning within the company. === Customer engagement === Online banking reduced face-to-face interaction between customers and their banks. Most banking transactions can now be conducted online or through mobile devices, rather than at a local branch with a teller. Social media provides a channel for firms to maintain personal contact with customers, replicating some of the interaction that was previously available at local branches. For example, a bank's Facebook page may feature an employee profile describing their job duties, which serves to present a more human face for larger institutions. === Lending === Social media is a core marketing channel for online peer-to-peer lending as well as small business lenders. Since these companies operate exclusively online, it makes sense for them to market online through social media channels. They are able to grow and find new lenders and buyers by utilizing social networks. === Trading === Social trading is an alternative way of analyzing financial data by looking at what other traders are doing and comparing, copying and discussing their techniques and strategies. Prior to the advent of social trading, investors and traders were relying on fundamental or technical analysis to form their investment decisions. Using social trading investors and traders could integrate into their investment decision-process social indicators from trading data-feeds of other traders. Investors also use platform like Reddit, Signal messaging or WeChat to create social communities to discuss investments and finance. In some cases they use this to join together using meme stocks to move financial markets, such as the 2021 GameStop short squeeze incident. They can also use social groups to launch and promote new products such as cryptocurrencies. Investing application like WeBull incorporate a forum style messaging system on each stock that is available for trading. Financial brokers such as Fidelity Investments, Interactive Brokers, and E-Trade have moved to incorporate community features in their investment apps. == Regulations == The use of social media by investors and financial services professionals for business purposes is subject to regulatory oversight, in the United States this is done primarily by the Financial Industry Regulatory Authority (FINRA). FINRA's rules, designed to protect investors from misleading information in all communications and this also applies to social media communications. This includes ensuring that social media posts follow financial advertising rules, such as being fair and balanced and not providing misleading information, and that advice is not provided by unqualified personnel, such as influencers and bank staff acting in a personal capacity. Financial firms have to maintain books and records of all interaction with customers and this includes social media. == New products and services == Social media has created entirely new products for the financial services sector, revolutionizing products and developing new industries through the merging of social technology and financial services. Fintech startups use social media to promote products to get them established. Several developing nations have used social media to leapfrog traditional financial technology; for example, WeChat Pay, which developed from the Chinese WeChat social media platform, became a major payment system in China within a few years. In 2015, according to consulting firm Accenture, 390 million people in China had registered to use mobile banking. This figure is more than the population of the United States. In the United States, the fintech company Venmo combines technology and financial services on a social platform. Other financial technology companies that have used social media to develop or promote financial products include: Lending Club – One of the first peer-to-peer lending businesses OnDeck Capital – A US online-only lending business Funding Circle – A UK-based online lending company Wise – A global online money transfers company Kabbage – A US online unsecured loan company later acquired by American Express Avant – A US online unsecured loan company Zopa – A UK online neobank providing peer-to-peer lending == Risks == === Reputational damage === Due to the real-time nature of social media, financial services companies can be impacted by potential reputational issues. Any negative experience by customers can easily be shared online and could become a viral phenomenon, those comments could likely have a detrimental effect on the company’s stock price and reputation. On the other hand, any positive experience a customer has can also be shared online. However, positive experiences are much less likely to become viral. === Scams === The nature of social media makes it easy to target individuals without being seen by the wider community, this allows scammers to target individuals. Example include romance scams such as the pig butchering scam where an individual is tricked to transfer funds or assets to the scammer over social media making it hard for law enforcement to track them or recover funds. === Customer privacy === Customer privacy is important for the financial services industry. It is critical that customer information such as a bank account numbers and other personal information is kept private. However, this information can be leaked if for example, a customer is unhappy with a bank’s service, they may tweet at the bank expressing their frustrations and include their name and account number.
Read more →
Brooklyn Bridge (software)

The Brooklyn Bridge from White Crane Systems was a data transfer enabler. Although it came with some hardware, it was the software which was the basis of the product. It also could transform the data's format. == Overview == The New York Times described its category as being among "communications packages used to transfer files." In an era of 300 baud, Brooklyn Bridge operated at "115,200 baud" so that a transfer which "at 300 baud took 4 minutes and 36 seconds" only needed 5 seconds. Unlike some communications packages, this one retains the original version-date, so as not to alarm people when they seem to have what looks like an update, when it's not. == Description == Once the software is installed, users comfortable with typing the word "COPY" can do so as readily as they sneakernet. An earlier review described it as "less cumbersome than conventional communications software" The use of neither specialized hardware nor specialized software is ideal in an era when this can be done using online or other "outside" services.
Read more →
Big memory

Big-memory computers are machines with a large amount of random-access memory (RAM). The computers are required for databases, graph analytics, or more generally, high-performance computing, data science, and big data. Some database systems called in-memory databases are designed to run mostly in memory, rarely if ever retrieving data from disk or flash memory. See list of in-memory databases. == Details == The performance of big-memory systems depends on how the central processing units (CPUs) access the memory, via a conventional memory controller or via non-uniform memory access (NUMA). Performance also depends on the size and design of the CPU cache. Performance also depends on operating system (OS) design. The huge pages feature in Linux and other OSes can improve the efficiency of virtual memory. The transparent huge pages feature in Linux can offer better performance for some big-memory workloads. The "Large-Page Support" in Microsoft Windows enables server applications to establish large-page memory regions which are typically three orders of magnitude larger than the native page size.
Read more →
Eigenmoments

EigenMoments is a set of orthogonal, noise robust, invariant to rotation, scaling and translation and distribution sensitive moments. Their application can be found in signal processing and computer vision as descriptors of the signal or image. The descriptors can later be used for classification purposes. It is obtained by performing orthogonalization, via eigen analysis on geometric moments. == Framework summary == EigenMoments are computed by performing eigen analysis on the moment space of an image by maximizing signal-to-noise ratio in the feature space in form of Rayleigh quotient. This approach has several benefits in Image processing applications: Dependency of moments in the moment space on the distribution of the images being transformed, ensures decorrelation of the final feature space after eigen analysis on the moment space. The ability of EigenMoments to take into account distribution of the image makes it more versatile and adaptable for different genres. Generated moment kernels are orthogonal and therefore analysis on the moment space becomes easier. Transformation with orthogonal moment kernels into moment space is analogous to projection of the image onto a number of orthogonal axes. Nosiy components can be removed. This makes EigenMoments robust for classification applications. Optimal information compaction can be obtained and therefore a few number of moments are needed to characterize the images. == Problem formulation == Assume that a signal vector s ∈ R n {\displaystyle s\in {\mathcal {R}}^{n}} is taken from a certain distribution having correlation C ∈ R n × n {\displaystyle C\in {\mathcal {R}}^{n\times n}} , i.e. C = E [ s s T ] {\displaystyle C=E[ss^{T}]} where E[.] denotes expected value. Dimension of signal space, n, is often too large to be useful for practical application such as pattern classification, we need to transform the signal space into a space with lower dimensionality. This is performed by a two-step linear transformation: q = W T X T s , {\displaystyle q=W^{T}X^{T}s,} where q = [ q 1 , . . . , q n ] T ∈ R k {\displaystyle q=[q_{1},...,q_{n}]^{T}\in {\mathcal {R}}^{k}} is the transformed signal, X = [ x 1 , . . . , x n ] T ∈ R n × m {\displaystyle X=[x_{1},...,x_{n}]^{T}\in {\mathcal {R}}^{n\times m}} a fixed transformation matrix which transforms the signal into the moment space, and W = [ w 1 , . . . , w n ] T ∈ R m × k {\displaystyle W=[w_{1},...,w_{n}]^{T}\in {\mathcal {R}}^{m\times k}} the transformation matrix which we are going to determine by maximizing the SNR of the feature space resided by q {\displaystyle q} . For the case of Geometric Moments, X would be the monomials. If m = k = n {\displaystyle m=k=n} , a full rank transformation would result, however usually we have m ≤ n {\displaystyle m\leq n} and k ≤ m {\displaystyle k\leq m} . This is specially the case when n {\displaystyle n} is of high dimensions. Finding W {\displaystyle W} that maximizes the SNR of the feature space: S N R t r a n s f o r m = w T X T C X w w T X T N X w , {\displaystyle SNR_{transform}={\frac {w^{T}X^{T}CXw}{w^{T}X^{T}NXw}},} where N is the correlation matrix of the noise signal. The problem can thus be formulated as w 1 , . . . , w k = a r g m a x w w T X T C X w w T X T N X w {\displaystyle {w_{1},...,w_{k}}=argmax_{w}{\frac {w^{T}X^{T}CXw}{w^{T}X^{T}NXw}}} subject to constraints: w i T X T N X w j = δ i j , {\displaystyle w_{i}^{T}X^{T}NXw_{j}=\delta _{ij},} where δ i j {\displaystyle \delta _{ij}} is the Kronecker delta. It can be observed that this maximization is Rayleigh quotient by letting A = X T C X {\displaystyle A=X^{T}CX} and B = X T N X {\displaystyle B=X^{T}NX} and therefore can be written as: w 1 , . . . , w k = a r g m a x x w T A w w T B w {\displaystyle {w_{1},...,w_{k}}={\underset {x}{\operatorname {arg\,max} }}{\frac {w^{T}Aw}{w^{T}Bw}}} , w i T B w j = δ i j {\displaystyle w_{i}^{T}Bw_{j}=\delta _{ij}} === Rayleigh quotient === Optimization of Rayleigh quotient has the form: max w R ( w ) = max w w T A w w T B w {\displaystyle \max _{w}R(w)=\max _{w}{\frac {w^{T}Aw}{w^{T}Bw}}} and A {\displaystyle A} and B {\displaystyle B} , both are symmetric and B {\displaystyle B} is positive definite and therefore invertible. Scaling w {\displaystyle w} does not change the value of the object function and hence and additional scalar constraint w T B w = 1 {\displaystyle w^{T}Bw=1} can be imposed on w {\displaystyle w} and no solution would be lost when the objective function is optimized. This constraint optimization problem can be solved using Lagrangian multiplier: max w w T A w {\displaystyle \max _{w}{w^{T}Aw}} subject to w T B w = 1 {\displaystyle {w^{T}Bw}=1} max w L ( w ) = max w ( w T A w − λ w T B w ) {\displaystyle \max _{w}{\mathcal {L}}(w)=\max _{w}(w{T}Aw-\lambda w^{T}Bw)} equating first derivative to zero and we will have: A w = λ B w {\displaystyle Aw=\lambda Bw} which is an instance of Generalized Eigenvalue Problem (GEP). The GEP has the form: A w = λ B w {\displaystyle Aw=\lambda Bw} for any pair ( w , λ ) {\displaystyle (w,\lambda )} that is a solution to above equation, w {\displaystyle w} is called a generalized eigenvector and λ {\displaystyle \lambda } is called a generalized eigenvalue. Finding w {\displaystyle w} and λ {\displaystyle \lambda } that satisfies this equations would produce the result which optimizes Rayleigh quotient. One way of maximizing Rayleigh quotient is through solving the Generalized Eigen Problem. Dimension reduction can be performed by simply choosing the first components w i {\displaystyle w_{i}} , i = 1 , . . . , k {\displaystyle i=1,...,k} , with the highest values for R ( w ) {\displaystyle R(w)} out of the m {\displaystyle m} components, and discard the rest. Interpretation of this transformation is rotating and scaling the moment space, transforming it into a feature space with maximized SNR and therefore, the first k {\displaystyle k} components are the components with highest k {\displaystyle k} SNR values. The other method to look at this solution is to use the concept of simultaneous diagonalization instead of Generalized Eigen Problem. === Simultaneous diagonalization === Let A = X T C X {\displaystyle A=X^{T}CX} and B = X T N X {\displaystyle B=X^{T}NX} as mentioned earlier. We can write W {\displaystyle W} as two separate transformation matrices: W = W 1 W 2 . {\displaystyle W=W_{1}W_{2}.} W 1 {\displaystyle W_{1}} can be found by first diagonalize B: P T B P = D B {\displaystyle P^{T}BP=D_{B}} . Where D B {\displaystyle D_{B}} is a diagonal matrix sorted in increasing order. Since B {\displaystyle B} is positive definite, thus D B > 0 {\displaystyle D_{B}>0} . We can discard those eigenvalues that large and retain those close to 0, since this means the energy of the noise is close to 0 in this space, at this stage it is also possible to discard those eigenvectors that have large eigenvalues. Let P ^ {\displaystyle {\hat {P}}} be the first k {\displaystyle k} columns of P {\displaystyle P} , now P T ^ B P ^ = D B ^ {\displaystyle {\hat {P^{T}}}B{\hat {P}}={\hat {D_{B}}}} where D B ^ {\displaystyle {\hat {D_{B}}}} is the k × k {\displaystyle k\times k} principal submatrix of D B {\displaystyle D_{B}} . Let W 1 = P ^ D B ^ − 1 / 2 {\displaystyle W_{1}={\hat {P}}{\hat {D_{B}}}^{-1/2}} and hence: W 1 T B W 1 = ( P ^ D B ^ − 1 / 2 ) T B ( P ^ D B ^ − 1 / 2 ) = I {\displaystyle W_{1}^{T}BW_{1}=({\hat {P}}{\hat {D_{B}}}^{-1/2})^{T}B({\hat {P}}{\hat {D_{B}}}^{-1/2})=I} . W 1 {\displaystyle W_{1}} whiten B {\displaystyle B} and reduces the dimensionality from m {\displaystyle m} to k {\displaystyle k} . The transformed space resided by q ′ = W 1 T X T s {\displaystyle q'=W_{1}^{T}X^{T}s} is called the noise space. Then, we diagonalize W 1 T A W 1 {\displaystyle W_{1}^{T}AW_{1}} : W 2 T W 1 T A W 1 W 2 = D A {\displaystyle W_{2}^{T}W_{1}^{T}AW_{1}W_{2}=D_{A}} , where W 2 T W 2 = I {\displaystyle W_{2}^{T}W_{2}=I} . D A {\displaystyle D_{A}} is the matrix with eigenvalues of W 1 T A W 1 {\displaystyle W_{1}^{T}AW_{1}} on its diagonal. We may retain all the eigenvalues and their corresponding eigenvectors since most of the noise are already discarded in previous step. Finally the transformation is given by: W = W 1 W 2 {\displaystyle W=W_{1}W_{2}} where W {\displaystyle W} diagonalizes both the numerator and denominator of the SNR, W T A W = D A {\displaystyle W^{T}AW=D_{A}} , W T B W = I {\displaystyle W^{T}BW=I} and the transformation of signal s {\displaystyle s} is defined as q = W T X T s = W 2 T W 1 T X T s {\displaystyle q=W^{T}X^{T}s=W_{2}^{T}W_{1}^{T}X^{T}s} . === Information loss === To find the information loss when we discard some of the eigenvalues and eigenvectors we can perform following analysis: η = 1 − t r a c e ( W 1 T A W 1 ) t r a c e ( D B − 1 / 2 P T A P D B − 1 / 2 ) = 1 − t r a c e ( D B ^ − 1 / 2 P ^ T A P ^ D B ^ − 1 / 2 ) t r a c e ( D B − 1 / 2 P T A P D B − 1 / 2 ) {\displaystyle {\begin{array}{lll}\eta &=&
Read more →
IEBus

IEBus (Inter Equipment Bus) is a communication bus specification "between equipments within a vehicle or a chassis" of Renesas Electronics. It defines OSI model layer 1 and layer 2 specification. IEBus is mainly used for car audio and car navigations, which established de facto standard in Japan, though SAE J1850 is major in United States. IEBus is also used in some vending machines, which major customer is Fuji Electric. Each button on the vending machine has an IEBus ID, i.e. has a controller. Detailed specification is disclosed to licensees only, but protocol analyzers are provided from some test equipment vendors. Its modulation method is PWM (Pulse-Width Modulation) with 6.00 MHz base clock originally, but most of automotive customers use 6.291 MHz, and physical layer is a pair of differential signalling harness. Its physical layer adopts half-duplex, asynchronous, and multi-master communication with carrier-sense multiple access with collision detection (CSMA/CD) for medium access control. It allows for up to fifty units on one bus over a maximum length of 150 meters. Two differential signalling lines are used with Bus+ / Bus− naming, sometimes labeled as Data(+) / Data(−). It is sometimes described as "IE-BUS", "IE-Bus," or "IE Bus," but these are incorrect. In formal, it is "IEBus." IEBus® and Inter Equipment Bus® are registered trademark symbols of Renesas Electronics Corporation, formerly NEC Electronics Corporation, (JPO: Reg. No.2552418 and 2552419, respectively). == History == In the middle of '80s, semiconductor unit of NEC Corporation, currently Renesas Electronics, started the study for increasing demands for automotive audio systems. IEBus is introduced as a solution for the distributed control system. In the late 1980s, several similar specifications, including the Domestic Digital Bus (D2B), the Japanese Home Bus (HBS), and the European Home System (EHS) are proposed by different companies or organizations. These were once discussed as IEC 61030, but it was withdrawn in 2006. IEBus is also a similar specification (refer to "Transfer signal format" section), but not listed in these criteria. As the result, IEBus becomes a de facto standard of car audio in Japan. Regarding the Domestic Digital Bus (D2B), it is re-defined as D2B Optical by Mercedes-Benz independently. As for Japanese Home Bus System (HBS), it is defined in 1988 as Home Bus System Standard Specification, ET-2101 by JEITA and REEA (Radio Engineering & Electronics Assiation) in Japan. It is being used by several Japanese air conditioner manufacturers (for example, M-Net from Mitsubishi and the P1/P2 or F1/F2 bus from Daikin). Fujitsu provided HBPC (Home Bus Protocol Controller) chip as MB86046B. But it is unclear whether Fujitsu (currently, Cypress) still manufactures this HBPC LSI as of 2018. Mitsumi Electric provides the MM1007 and MM1192 driver ICs for HBS. The HBS specification is also discussed in the Echonet Consortium. In 2014, a utility model patent for protocol converter from HBS to RS-485 is granted in China as "CN204006496U." Regarding the replacement of IEBus, a paper by Hyundai Autonet, currently Hyundai Mobis, describes as follows. "In communication methods for digital input capable amplifiers, Inter Equipment Bus (IEBus) was used in early times, but for now, Controller Area Network (CAN) is mainly used." == Protocol overview == A master talks to a slave. Each unit has a master and a slave address register. Only one device can talk on the bus at any given time. There is a pecking order for the types of communications which will take precedence over another. Each communication from master to slave must be replied to by the slave going back to the master with acknowledge bits each of those show ACK or NAK. If the master does not receive the ACK within a predefined time allowance for a mode, it drops the communication and returns to its standby (listen) mode. Detailed specification of OSI model layer 2 is disclosed to licensees only, but protocol analyzers are provided from some test equipment vendors. In 2012, one of Chinese manufacturer's patent is granted as "CN202841169U". An open-source software emulator called "IEBus Studio" exists on a repository of SourceForge, but the last update was on 2008-02-24. Another open-source analyzer software called "IEBusAnalyzer" is available on GitHub repository. Some hobbyist made some tools also. === Physical layer (OSI model layer 1) specification overview === From μPD6708 data sheet. and μPD78098B Subseries user's manual, hardware. Communication system Half-duplex asynchronous communication Multi-master system All the units connected to the IEBus can transfer data to the other units. Broadcast communication function (communication between one unit and multiple units) Normally, communication is individually carried out from one unit to another. By using the broadcast communication function, however, communication can be executed from one unit to plural units as follows: Group broadcast communication: Broadcast communication to group units Simultaneous broadcast communication: Broadcast communication to all units Effective transmission rate The effective transmission rate can be selected from the following three communication modes: Mixture of the plural of modes in the same bus line is not allowed. Correct communication between different base clock is not possible. Access control CSMA/CD (Carrier Sense Multiple Access with Collision Detection) The priority of occupying IEBus is as follows: «1» Broadcast communication takes precedence over individual communication. «2» The lower the master address, the higher the priority. Communication scale Number of units: 50 MAX. Cable length: 150 m MAX. (when a twisted pair cable is used) Load capacity: MAX. 8000 pF; between Bus+ and Bus−, (6.000000 MHz base clock) MAX. 7100 pF; between Bus+ and Bus−, (6.291456 MHz base clock) Terminating resistor: 120 Ω Logic level Logic 1: Low level. Voltage difference between Bus+ and Bus− is under 20mV Logic 0: High Level. Voltage difference between Bus+ and Bus− is over 120mV In-phase input voltage high: Bus+ ≤ (VDD-1.0) V, Bus− ≥ 1.0 V === Transfer signal format === From μPD6708 data sheet. and μPD78098B Subseries user's manual, hardware. This frame format is much similar to that of Domestic Digital Bus (D2B). All fields are MSB first. ==== Functions of Control bits ==== === Bit format === Each IEBus bit consists of four periods. Preparation period: The first or subsequent low-level (logic "1") period Synchronization period: Next high-level (logic "0") period Data period: Period indicating value of bit; ether low-level (logic "1") or high-level (logic "0") Stop period: The last low-level (logic "1") period Synchronization is done by each bit. Time lengths of the synchronization period and data period are almost the same. The time of the entire bits' and each bit's specification, related to the time of each period allocated to it, differ depending both on the type of the transmit bit and on whether the unit is the master or a slave unit. == Automotive manufacturers using IEBus == Each manufacturer has its own name, but it is not an alias of IEBus. Those are specifications of wire harness which comprise control cables based on IEBus, OSI model layer 3 and above communication protocol, audio cables, interconnection couplers, and so on. === Pioneer === Pioneer Corporation employed IEBus for its original branded car audio in early '90s. In its earlier stage, it was used just for control bus between the head unit in dashboard and the CD changer usually placed in trunk room. Nowadays, the specification includes connection between head units, navigation systems, rear speaker systems, and so on. IP-Bus: Wire harness specification. === Toyota === Pioneer Corporation pushed Toyota Motor Corporation to adopt IEBus as the genuine parts. In 1994, Toyota decided to employ IEBus for its genuine specification, but it is slightly different from that of Pioneer. It is named as AVC-LAN. AVC-LAN: Wire harness specification, based on mode 2. === Honda/Acura === Pioneer Corporation also pushed Honda Motor. Honda also decided to adopt IEBus as its genuine parts specification just after Toyota do so. GA-NET II: Wire harness specification. Honda Music Link: Honda genuine gadget to connect Apple Inc. products. A hobbyist made touch screen controller on Acura TSX for a Car PC installed in the trunk. === Sirius XM Satellite Radio === Sirius XM Satellite Radio is a satellite broadcasting radio operator in US. Its digital media receiver equipment utilizes IEBus. == Evaluation boards == === SAKURA board === GR-SAKUKRA board and GR-SAKURA-FULL board are Renesas official promotion boards of RX63N chip, which enables IEBus mode 0 and 1, but not mode 2, i.e. not available for Toyota AVC-LAN. They are an Arduino pin compatible low-price ones, suitable for hobbyists. Their color of printed circuit board is SAKURA in Japanese, which means cherry blossom. To e
Read more →
Big memory

Big-memory computers are machines with a large amount of random-access memory (RAM). The computers are required for databases, graph analytics, or more generally, high-performance computing, data science, and big data. Some database systems called in-memory databases are designed to run mostly in memory, rarely if ever retrieving data from disk or flash memory. See list of in-memory databases. == Details == The performance of big-memory systems depends on how the central processing units (CPUs) access the memory, via a conventional memory controller or via non-uniform memory access (NUMA). Performance also depends on the size and design of the CPU cache. Performance also depends on operating system (OS) design. The huge pages feature in Linux and other OSes can improve the efficiency of virtual memory. The transparent huge pages feature in Linux can offer better performance for some big-memory workloads. The "Large-Page Support" in Microsoft Windows enables server applications to establish large-page memory regions which are typically three orders of magnitude larger than the native page size.
Read more →
Business intelligence

Business intelligence (BI) consists of strategies, methodologies, and technologies used by enterprises for data analysis and management of business information to inform business strategies and business operations. Common functions of BI technologies include reporting, online analytical processing, analytics, dashboard development, data mining, process mining, complex event processing, business performance management, benchmarking, text mining, predictive analytics, and prescriptive analytics. BI tools can handle large amounts of structured and sometimes unstructured data to help organizations identify, develop, and otherwise create new strategic business opportunities. They aim to allow for the easy interpretation of these big data. Identifying new opportunities and implementing an effective strategy based on insights is assumed to potentially provide businesses with a competitive market advantage and long-term stability, and help them take strategic decisions. Business intelligence can be used by enterprises to support a wide range of business decisions ranging from operational to strategic. Basic operating decisions include product positioning or pricing. Strategic business decisions involve priorities, goals, and directions at the broadest level. In all cases, business intelligence is considered most effective when it combines data from the market in which a company operates (external data) with data from internal company sources, such as financial and operational information. When integrated, external and internal data provide a comprehensive view that creates ‘intelligence’ not possible from any single data source alone. Among their many uses, business intelligence tools empower organizations to gain insight into new markets, to assess demand and suitability of products and services for different market segments, and to gauge the impact of marketing efforts. BI applications use data gathered from a data warehouse (DW) or from a data mart, and the concepts of BI and DW combine as "BI/DW" or as "BIDW". A data warehouse contains a copy of analytical data that facilitates decision support. == History == The earliest known use of the term business intelligence is in Richard Millar Devens' Cyclopædia of Commercial and Business Anecdotes (1865). Devens used the term to describe how the banker Sir Henry Furnese gained profit by receiving and acting upon information about his environment, prior to his competitors: Throughout Holland, Flanders, France, and Germany, he maintained a complete and perfect train of business intelligence. The news of the many battles fought was thus received first by him, and the fall of Namur added to his profits, owing to his early receipt of the news. The ability to collect and react accordingly based on the information retrieved, Devens says, is central to business intelligence. When Hans Peter Luhn, a researcher at IBM, used the term business intelligence in an article published in 1958, he employed the Webster's Dictionary definition of intelligence: "the ability to apprehend the interrelationships of presented facts in such a way as to guide action towards a desired goal." In 1989, Howard Dresner (later a Gartner analyst) proposed business intelligence as an umbrella term to describe "concepts and methods to improve business decision making by using fact-based support systems." It was not until the late 1990s that this usage was widespread. == Definition == According to Solomon Negash and Paul Gray, business intelligence (BI) can be defined as systems that combine: Data gathering Data storage Knowledge management with analysis to evaluate complex corporate and competitive information for presentation to planners and decision makers, with the objective of improving the timeliness and the quality of the input to the decision process." According to Forrester Research, business intelligence is "a set of methodologies, processes, architectures, and technologies that transform raw data into meaningful and useful information used to enable more effective strategic, tactical, and operational insights and decision-making." Under this definition, business intelligence encompasses information management (data integration, data quality, data warehousing, master-data management, text- and content-analytics, et al.). Therefore, Forrester refers to data preparation and data usage as two separate but closely linked segments of the business-intelligence architectural stack. Some elements of business intelligence are: Multidimensional aggregation and allocation Denormalization, tagging, and standardization Realtime reporting with analytical alert A method of interfacing with unstructured data sources Group consolidation, budgeting, and rolling forecasts Statistical inference and probabilistic simulation Key performance indicators optimization Version control and process management Open item management Forrester distinguishes this from the business-intelligence market, which is "just the top layers of the BI architectural stack, such as reporting, analytics, and dashboards." === Compared with competitive intelligence === Though the term business intelligence is sometimes a synonym for competitive intelligence (because they both support decision making), BI uses technologies, processes, and applications to analyze mostly internal, structured data and business processes while competitive intelligence gathers, analyzes, and disseminates information with a topical focus on company competitors. If understood broadly, competitive intelligence can be considered as a subset of business intelligence. === Compared with business analytics === Business intelligence and business analytics are sometimes used interchangeably, but there are alternate definitions. Thomas Davenport, professor of information technology and management at Babson College argues that business intelligence should be divided into querying, reporting, Online analytical processing (OLAP), an "alerts" tool, and business analytics. In this definition, business analytics is the subset of BI focusing on statistics, prediction, and optimization, rather than the reporting functionality. == Unstructured data == Business operations can generate a very large amount of data in the form of emails, memos, notes from call centers, news, user groups, chats, reports, web pages, presentations, image files, video files, and marketing material. According to Merrill Lynch, more than 85% of all business information exists in these forms; a company might only use such a document a single time. Because of the way it is produced and stored, this information is either unstructured or semi-structured. The management of semi-structured data is an unsolved problem in the information technology industry. According to projections from Gartner (2003), white-collar workers spend 30–40% of their time searching, finding, and assessing unstructured data. BI uses both structured and unstructured data. The former is easy to search, and the latter contains a large quantity of the information needed for analysis and decision-making. Because of the difficulty of properly searching, finding, and assessing unstructured or semi-structured data, organizations may not draw upon these vast reservoirs of information, which could influence a particular decision, task, or project. This can ultimately lead to poorly informed decision-making. Therefore, when designing a business intelligence/DW solution, the specific problems associated with semi-structured and unstructured data must be accommodated, as well as those associated with structured data. === Limitations of semi-structured and unstructured data === There are several challenges to developing BI with semi-structured data. According to Inmon & Nesavich, some of those are: Physically accessing unstructured textual data – unstructured data is stored in a huge variety of formats. Terminology – Among researchers and analysts, there is a need to develop standardized terminology. Volume of data – As stated earlier, up to 85% of all data exists as semi-structured data. Couple that with the need for word-to-word and semantic analysis. Searchability of unstructured textual data – A simple search on some data, e.g. apple, results in links where there is a reference to that precise search term. (Inmon & Nesavich, 2008) gives an example: "a search is made on the term felony. In a simple search, the term felony is used, and everywhere there is a reference to felony, a hit to an unstructured document is made. But a simple search is crude. It does not find references to crime, arson, murder, embezzlement, vehicular homicide, and such, even though these crimes are types of felonies". === Metadata === To solve problems with searchability and assessment of data, it is necessary to know something about the content. This can be done by adding context through the use of metadata. Many systems already capture some metadata (e.g. filename, author, size, etc.), but more usef
Read more →
Relational data mining

Relational data mining is the data mining technique for relational databases. Unlike traditional data mining algorithms, which look for patterns in a single table (propositional patterns), relational data mining algorithms look for patterns among multiple tables (relational patterns). For most types of propositional patterns, there are corresponding relational patterns. For example, there are relational classification rules (relational classification), relational regression tree, and relational association rules. There are several approaches to relational data mining: Inductive Logic Programming (ILP) Statistical Relational Learning (SRL) Graph Mining Propositionalization Multi-view learning == Algorithms == Multi-Relation Association Rules: Multi-Relation Association Rules (MRAR) is a new class of association rules which in contrast to primitive, simple and even multi-relational association rules (that are usually extracted from multi-relational databases), each rule item consists of one entity but several relations. These relations indicate indirect relationship between the entities. Consider the following MRAR where the first item consists of three relations live in, nearby and humid: “Those who live in a place which is near by a city with humid climate type and also are younger than 20 -> their health condition is good”. Such association rules are extractable from RDBMS data or semantic web data. == Software == Safarii: a Data Mining environment for analysing large relational databases based on a multi-relational data mining engine. Dataconda: a software, free for research and teaching purposes, that helps mining relational databases without the use of SQL. == Datasets == Relational dataset repository: a collection of publicly available relational datasets.
Read more →
HTTP Strict Transport Security

HTTP Strict Transport Security (HSTS) is a policy mechanism that helps to protect websites against man-in-the-middle attacks such as protocol downgrade attacks and cookie hijacking. It allows web servers to declare that web browsers (or other complying user agents) should automatically interact with it using only HTTPS connections, which provide Transport Layer Security (TLS/SSL), unlike the insecure HTTP used alone. HSTS is an IETF standards track protocol and is specified in RFC 6797. The HSTS Policy is communicated by the server to the user agent via an HTTP response header field named Strict-Transport-Security. HSTS Policy specifies a period of time during which the user agent should only access the server in a secure fashion. Websites using HSTS often do not accept clear text HTTP, either by rejecting connections over HTTP or systematically redirecting users to HTTPS (though this is not required by the specification). The consequence of this is that a user-agent not capable of doing TLS will not be able to connect to the site. The protection normally only applies after a user has visited the site at least once, relying on the principle of "trust on first use". The way this protection works is that when a user entering or selecting an HTTP (not HTTPS) URL to the site, the client, such as a Web browser, will automatically upgrade to HTTPS without making an HTTP request, thereby preventing any HTTP man-in-the-middle attack from occurring. To counteract this problem, an HSTS preload list maintained by Google Chrome and used by other major web browsers is maintained. If a domain is on this list, the browser skips the initial request and encrypts all communication immediately. Additional domains can be registered at no cost. == Specification history == The HSTS specification was published as RFC 6797 on 19 November 2012 after being approved on 2 October 2012 by the IESG for publication as a Proposed Standard RFC. The authors originally submitted it as an Internet Draft on 17 June 2010. With the conversion to an Internet Draft, the specification name was altered from "Strict Transport Security" (STS) to "HTTP Strict Transport Security", because the specification applies only to HTTP. The HTTP response header field defined in the HSTS specification however remains named "Strict-Transport-Security". The last so-called "community version" of the then-named "STS" specification was published on 18 December 2009, with revisions based on community feedback. The original draft specification by Jeff Hodges from PayPal, Collin Jackson, and Adam Barth was published on 18 September 2009. The HSTS specification is based on original work by Jackson and Barth as described in their paper "ForceHTTPS: Protecting High-Security Web Sites from Network Attacks". Additionally, HSTS is the realization of one facet of an overall vision for improving web security, put forward by Jeff Hodges and Andy Steingruebl in their 2010 paper The Need for Coherent Web Security Policy Framework(s). == HSTS mechanism overview == A server implements an HSTS policy by supplying a header over an HTTPS connection (HSTS headers over HTTP are ignored). For example, a server could send a header such that future requests to the domain for the next year (max-age is specified in seconds; 31,536,000 is equal to one non-leap year) use only HTTPS: Strict-Transport-Security: max-age=31536000. When a web application issues HSTS Policy to user agents, conformant user agents behave as follows: Automatically turn any insecure links referencing the web application into secure links (e.g. http://example.com/some/page/ will be modified to https://example.com/some/page/ before accessing the server). If the security of the connection cannot be ensured (e.g. the server's TLS certificate is not trusted), the user agent must terminate the connection and should not allow the user to access the web application. This helps protect web application users against some passive (eavesdropping) and active network attacks. A man-in-the-middle attacker has a greatly reduced ability to intercept requests and responses between a user and a web application server while the user's browser has HSTS Policy in effect for that web application. == Applicability == The most important security vulnerability that HSTS can fix is SSL-stripping man-in-the-middle attacks, first publicly introduced by Moxie Marlinspike in his 2009 BlackHat Federal talk "New Tricks For Defeating SSL In Practice". The SSL (and TLS) stripping attack works by transparently converting a secure HTTPS connection into a plain HTTP connection. The user can see that the connection is insecure, but crucially there is no way of knowing whether the connection should be secure. At the time of Marlinspike's talk, many websites did not use TLS/SSL, therefore there was no way of knowing (without prior knowledge) whether the use of plain HTTP was due to an attack, or simply because the website had not implemented TLS/SSL. Additionally, no warnings are presented to the user during the downgrade process, making the attack fairly subtle to all but the most vigilant. Marlinspike's sslstrip tool, presented at Black Hat DC 2009, fully automates the attack. HSTS addresses this problem by informing the browser that connections to the site should always use TLS/SSL. The HSTS header can be stripped by the attacker if this is the user's first visit. Google Chrome, Mozilla Firefox, Internet Explorer, and Microsoft Edge attempt to limit this problem by including a "pre-loaded" list of HSTS sites. Unfortunately this solution cannot scale to include all websites on the internet. See limitations, below. HSTS can also help to prevent having one's cookie-based website login credentials stolen by widely available tools such as Firesheep. Because HSTS is time limited, it is sensitive to attacks involving shifting the victim's computer time e.g. using false NTP packets. == Limitations == The initial request remains unprotected from active attacks if it uses an insecure protocol such as plain HTTP or if the URI for the initial request was obtained over an insecure channel. The same applies to the first request after the activity period specified in the advertised HSTS Policy max-age (sites should set a period of several days or months depending on user activity and behavior). === Solutions with preload list === Google Chrome, Mozilla Firefox, and Internet Explorer/Microsoft Edge address this limitation by implementing a "HSTS preloaded list", which is a list that contains known sites supporting HSTS. This list is distributed with the browser so that it uses HTTPS for the initial request to the listed sites as well. As previously mentioned, these pre-loaded lists cannot scale to cover the entire Web. A potential solution might be achieved by using DNS records to declare HSTS Policy, and accessing them securely via DNSSEC, optionally with certificate fingerprints to ensure validity (which requires running a validating resolver to avoid last mile issues). Junade Ali has noted that HSTS is ineffective against the use of false domains; by using DNS-based attacks, it is possible for a man-in-the-middle interceptor to serve traffic from an artificial domain which is not on the HSTS Preload list, this can be made possible by DNS Spoofing Attacks, or simply a domain name that misleadingly resembles the real domain name such as www.example.org instead of www.example.com. Even with an HSTS preloaded list, HSTS cannot prevent advanced attacks against TLS itself, such as the BEAST or CRIME attacks introduced by Juliano Rizzo and Thai Duong. Attacks against TLS itself are orthogonal to HSTS policy enforcement. Neither can it protect against attacks on the server - if someone compromises it, it will happily serve any content over TLS. === Privacy issues === HSTS can be used to near-indelibly tag visiting browsers with recoverable identifying data (supercookies) which can persist in and out of browser "incognito" privacy modes. By creating a web page that makes multiple HTTP requests to selected domains, for example, if twenty browser requests to twenty different domains are used, theoretically over one million visitors can be distinguished (220) due to the resulting requests arriving via HTTP vs. HTTPS; the latter being the previously recorded binary "bits" established earlier via HSTS headers. == Browser support == Chromium and Google Chrome since version 4.0.211.0 Firefox since version 4; with Firefox 17, Mozilla integrates a list of websites supporting HSTS. Opera since version 12 Safari since OS X Mavericks (version 10.9, late 2013) Internet Explorer 11 on Windows 8.1 and Windows 7 with KB3058515 installed (Released as a Windows Update in June 2015) Microsoft Edge and Internet Explorer 11 on Windows 10 BlackBerry 10 Browser and WebView since BlackBerry OS 10.3.3. == Deployment best practices == Depending on the actual deployment there are certain threats (e.g. cookie injection attacks) t
Read more →
Classora

Classora is a knowledge base for the Internet oriented to data analysis. From a practical point of view, Classora is a digital repository that stores structured information and allows it to be displayed in multiple formats: analytically, graphically, geographically (through maps); as well as carry out OLAP analysis. The information contained in Classora comes from public sources and is uploaded into the system through bots and ETL processes. The Knowledge Base has a commercial API for semantic enhancement, and an open web through which any user can access to part of the information collected (it also allows users to complete data and share opinions). Internally, Classora is organized into Knowledge Units and Reports. A «Knowledge Unit» is any element of the World about which information may be stored and presented in the form of a data sheet (a person, a company, a country, etc.) A «Report» is a group of Knowledge Units: a ranking of companies, a sport classification table, a survey about people, etc. In fact, one of the technical capabilities of Classora is that it allows the comparison of reports and knowledge units gathered from different sources, thereby generating an added value for the media in which this information is published: digital media, interactive TV, etc. == Key definitions == === Knowledge unit === The units of knowledge (also known as entries) in Classora are data sheets that have a certain semantic equivalence with the articles on the Wikipedia: they store information about any element of the world, be it a film, a country, a company or an animal. However, they differ from Wikipedia in that Classora stores structured information, enriched with a metadata layer; and therefore it is able to automatically interpret the meaning of each unit of knowledge. === Data report === A report is a group of units of knowledge in which the repetition of elements is not allowed. This definition includes any list, poll, ranking, etc.; and, in general, any consultation that involves more than one unit of knowledge. Classora excels at the reports management due to its visualization capabilities, being able to display data in the form of tables, graphs and maps. Types of reports: Sports scores: Sports competitions results sanctioned by the competent institution. Rankings and lists: All types of interesting and curious lists, whether they have an implicit order or not. Polls: Units of knowledge that are ranked according to users’ votes. Queries to the Knowledge Base: Questions from users using CQL. Networks of connections: automatically calculated from the reports and the taxonomy of each Knowledge Unit. === Organizational taxonomy === An organizational taxonomy (also referred to as entry type) is a data sheet that brings together the common attributes of a set of units of knowledge. For instance, the organizational taxonomy F1 Driver displays attributes such as date of debut, team, etc.; and the organizational taxonomy Football Club presents attributes such as city, stadium, etc. In Classora, taxonomies are hierarchically organized, so that they inherit attributes from their parent taxonomies. For instance, F1 Driver is a subsidiary taxonomy of Sportsperson, which is a subsidiary taxonomy of Person, which in turn is a subsidiary taxonomy of Organism. The simplest type of entry in Classora is Classora Object. All the other taxonomies are its subsidiaries and inherit its attributes. In fact, the only attribute Classora Object possesses is name (all units of knowledge are required to have one name at least). == Architecture of Classora == === Data Extraction Module === The Data Extraction Module consists of a set of robots coordinated by software that also manages the potential incidents. Most of the information available in Classora is automatically uploaded through those robots, which connect to the main online public sources to gather all types of data. There are three categories of robots: Extraction robots: responsible for the massive uploading of reports from official public sources (FIFA, CIA, IMF, Eurostat...). They are used for either absolute or incremental data uploading. Data scanner robots: responsible for looking for and updating the data of a unit of knowledge. They use specific sources to perform this task: Wikipedia, IMDB, World Bank, etc. Content aggregators: they don’t connect to external sources. Instead, they generate new information using Classora’s internal database. === Participatory Module === In Classora’s Open Website, Internet users may participate providing their knowledge as they would on the Wikipedia. There are different ways to participate: adding or correcting data in the Knowledge Base, voting in surveys (participatory rankings) and creating new Knowledge Units and Data Reports. === Connectivity Module === The Knowledge Base is designed to be embedded in multi-platform, multi-channel systems, thus enabling its integration into mobile devices, tablets, interactive TV, etc. This integration may be carried out through specific plugins (for navigators or other devices) or an API REST that provides content in XML or JSON formats. The API is divided into three blocks of operations. The first one is the block of general utility tools (ranging from autosuggest components about geographical hierarchies to operations to obtain the list of today’s celebrity birthdays, using CQL). The second one is the block of operations for widget generation (graphs, maps, rankings) using information from the knowledge base. Finally, there is a block of operations designed for the publication of free-source content. == Project statistics == As of April 2012, 2,000,000 Knowledge Units, 15,000 Reports, around 10,000 Maps and several million potential Comparative Analyses had been added to Classora. According to the site of web metrics Alexa, Classora Open Website is ranked at 100,557 globally and at 2,880 in the Spanish traffic ranking. Users spend an average of 9 ½ minutes in Classora.
Read more →