CiteAgent Trajectories

You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you s... You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you select really corresponds to the excerpt: there will be details mentioned in the excerpt that should appear in the paper. If you read an abstract and it seems like it could be the paper we’re looking for, read the paper to make sure. Also: sometimes you’ll read a paper that cites the paper we’re looking for. In such cases, please go to the references in order to find the full name of the paper we’re looking for, and search for it, and then select it. This technique of lower bounding mutual information is known as Variational Information Maximization [CITATION]. You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you select really corresponds to the excerpt: there will be details mentioned in the excerpt that should appear in the paper. If you read an abstract and it seems like it could be the paper we’re looking for, read the paper to make sure. Also: sometimes you’ll read a paper that cites the paper we’re looking for. In such cases, please go to the references in order to find the full name of the paper we’re looking for, and search for it, and then select it. This technique of lower bounding mutual information is known as Variational Information Maximization [CITATION].

- Paper ID: 7c27224be9b4fbe9a9546fd5922afcfe8ef3073f Title: Variational Information Maximization for Feature Selection Abstract: Featu... - Paper ID: 7c27224be9b4fbe9a9546fd5922afcfe8ef3073f Title: Variational Information Maximization for Feature Selection Abstract: Feature selection is one of the most fundamental problems in machine learning. An extensive body of work on information-theoretic feature selection exists which is based on maximizing mutual information between subsets of features and class labels. Practical methods are forced to rely on approximations due to the difficulty of estimating mutual information. We demonstrate that approximations made by existing methods are based on unrealistic assumptions. We formulate a more flexible and general class of assumptions based on variational distributions and use them to tractably generate lower bounds for mutual information. These bounds define a novel information-theoretic framework for feature selection, which we prove to be optimal under tree graphical models with proper choice of variational distributions. Our experiments demonstrate that the proposed met... - Paper ID: 7c27224be9b4fbe9a9546fd5922afcfe8ef3073f Title: Variational Information Maximization for Feature Selection Abstract: Feature selection is one of the most fundamental problems in machine learning. An extensive body of work on information-theoretic feature selection exists which is based on maximizing mutual information between subsets of features and class labels. Practical methods are forced to rely on approximations due to the difficulty of estimating mutual information. We demonstrate that approximations made by existing methods are based on unrealistic assumptions. We formulate a more flexible and general class of assumptions based on variational distributions and use them to tractably generate lower bounds for mutual information. These bounds define a novel information-theoretic framework for feature selection, which we prove to be optimal under tree graphical models with proper choice of variational distributions. Our experiments demonstrate that the proposed method strongly outperforms existing information-theoretic feature selection approaches. Citation Count: 47 - Paper ID: 77c751cbae9de37cc0a8aaf024cb172da0fc6bea Title: Variational Information Maximization in Stochastic Environments Abstract: Information maximization is a common framework of unsupervised learning, which may be used for extracting informative representations y of the observed patterns x. The key idea there is to maximize mutual information (MI), which is a formal measure of coding efficiency. Unfortunately, exact maximization of MI is computationally tractable only in a few special cases; more generally, approximations need to be considered. Here we describe a family of variational lower bounds on mutual information which gives rise to a formal and theoretically rigorous approach to information maximization in large-scale stochastic channels. We hope that the results presented in this work are potentially interesting for maximizing mutual information from several perspectives. First of all, our method optimizes a proper lower bound, rather than a surrogate objective criterion or an approximation of MI (which may only be accurate under specific asymptotic assumptions, and weak or even undefined when the assumptions are violated). Secondly, the flexibility of the choice of the variational distribution makes it possible to generalize and improve simple bounds on MI. For example, we may introduce tractable auxiliary variational bounds on MI, which may be used to improve on any simple generic approach without altering properties of the original channel. Thirdly, the suggested variational framework is typically simpler than standard variational approaches to maximizing the conditional likelihood in stochastic autoencoder models, while it leads to the same fixed points in its simplest formulation; this gives rise to more efficient optimization procedures. Finally, in some cases the variational framework results in optimization procedures which only require local computations, which may be particularly attractive from the neuro-biological perspective. Possibly the most important contribution of this work is a rigorous and general framework for maximizing the mutual information in intrinsically intractable channels. We show that it gives rise to simple, stable, and easily generalizable optimization procedures, which outperform and supersede many of the common approximate information-maximizing techniques. We demonstrate our results by considering clustering, dimensionality reduction, and binary stochastic coding problems, and discuss a link to approximate statistical inference. Citation Count: 9 - Paper ID: f439525f95c5e16eb1474926aecc1d659cce49b3 Title: Variational Information Maximization for Neural Coding Abstract: None Citation Count: 14 - Paper ID: 5e42ed1b44ceeafaa726e6b7b1f89ea46d57f581 Title: Auxiliary Variational Information Maximization for Dimensionality Reduction Abstract: None Citation Count: 2 - Paper ID: 2b8ae32585fb5fe432d44798199e2edb0f3d1df6 Title: Variational Information Maximization in Gaussian Channels Abstract: Recently, we introduced a simple variational bound on mutual information, that resolves some of the difficulties in the application of information theory to machine learning. Here we study a specific application to Gaussian channels. It is well known that PCA may be viewed as the solution to maximizing information transmission between a high dimensional vector and its low dimensional representation . However, such results are based on assumptions of Gaussianity of the sources. In this paper, we show how our mutual information bound, when applied to this arena, gives PCA solutions, without the need for the Gaussian assumption. Furthermore, it naturally generalizes to providing an objective function for Kernel PCA, enabling the principled selection of kernel parameters. Citation Count: 1 - Paper ID: 9c2122a4bc9041a65a45610878e7884b766f89bf Title: Variational Information Maximization for Population Coding Abstract: The goal of neural processing assemblies is varied, and in many cases still rather unclear. However, a possibly reasonable subgoal is that sensory information may be encoded efficiently in a population of neurons. In this context, Mutual Information is a long studied measure of coding efficiency, and many attempts to apply this to {\em population coding} have been made. However, this is a numerically intractable task, and most previous studies redefine the criterion in forms of an approximation to Mutual Information, the Fisher Information being one such well-known approach. Here we describe a principled bound maximisation procedure for Mutual Information learning of population codes in a simple point neural model, and compare it with other approaches. Citation Count: 1 - Paper ID: b113a8e4076c39a07683678bfd8f185830c0b86a Title: Variational Information Maximization and ( K ) PCA Abstract: Recently, we introduced a simple variational bound on mutual information, that resolves some of the difficulties in the application of information theory to machine learning. Here we study a specific application to Gaussian channels. It is well known that PCA may be viewed as the solution to maximizing information transmission between a high dimensional vector x and its low dimensional representation y. However, such results are based on assumptions of Gaussianity of the sources x. In this paper, we show how our mutual information bound, when applied to this arena, gives PCA solutions, without the need for the Gaussian assumption. Furthermore, it naturally generalizes to providing an objective function for Kernel PCA, enabling the principled selection of kernel parameters. Citation Count: 0 - Paper ID: 317cd4522b1f4a6f889743578143bb8823623f8b Title: VIME: Variational Information Maximizing Exploration Abstract: Scalable and effective exploration remains a key challenge in reinforcement learning (RL). While there are methods with optimality guarantees in the setting of discrete state and action spaces, these methods cannot be applied in high-dimensional deep RL scenarios. As such, most contemporary RL relies on simple heuristics such as epsilon-greedy exploration or adding Gaussian noise to the controls. This paper introduces Variational Information Maximizing Exploration (VIME), an exploration strategy based on maximization of information gain about the agent's belief of environment dynamics. We propose a practical implementation, using variational inference in Bayesian neural networks which efficiently handles continuous state and action spaces. VIME modifies the MDP reward function, and can be applied with several different underlying RL algorithms. We demonstrate that VIME achieves significantly better performance compared to heuristic exploration methods across a variety of continuous control tasks and algorithms, including tasks with very sparse rewards. Citation Count: 697 - Paper ID: aae4efb3d412d585ea0dec03f933397c93caf989 Title: The IM algorithm: a variational approach to Information Maximization Abstract: The maximisation of information transmission over noisy channels is a common, albeit generally computationally difficult problem. We approach the difficulty of computing the mutual information for noisy channels by using a variational approximation. The resulting IM algorithm is analagous to the EM algorithm, yet maximises mutual information, as opposed to likelihood. We apply the method to several practical examples, including linear compression, population encoding and CDMA. Citation Count: 398 - Paper ID: 8d410829db1d6d3f33977d5d58a18805e187f547 Title: Information Maximization in Noisy Channels : A Variational Approach Abstract: The maximisation of information transmission over noisy channels is a common, albeit generally computationally difficult problem. We approach the difficulty of computing the mutual information for noisy channels by using a variational approximation. We apply the method to several practical examples, including linear compression, population encoding and CDMA. We demonstrate that our approach enables one to calculate encoding and decoding schemes that can be optimised in a principled manner. Citation Count: 28

Variational Information Maximization for Feature Selection Shuyang Gao Greg Ver Steeg Aram Galstyan University of Southern California, Infor... Variational Information Maximization for Feature Selection Shuyang Gao Greg Ver Steeg Aram Galstyan University of Southern California, Information Sciences Institute gaos@usc.edu ,gregv@isi.edu ,galstyan@isi.edu Abstract Feature selection is one of the most fundamental problems in machine learning. An extensive body of work on information-theoretic feature selection exists which is based on maximizing mutual information between subsets of features and class labels. Practical methods are forced to rely on approximations due to the difficulty of estimating mutual information. We demonstrate that approximations made by existing methods are based on unrealistic assumptions. We formulate a more flex- ible and general class of assumptions based on variational distributions and use them to tractably generate lower bounds for mutual information. These bounds define a novel information-theoretic framework for feature selection, which we prove to be optimal under tree graphical models with proper c... Variational Information Maximization for Feature Selection Shuyang Gao Greg Ver Steeg Aram Galstyan University of Southern California, Information Sciences Institute gaos@usc.edu ,gregv@isi.edu ,galstyan@isi.edu Abstract Feature selection is one of the most fundamental problems in machine learning. An extensive body of work on information-theoretic feature selection exists which is based on maximizing mutual information between subsets of features and class labels. Practical methods are forced to rely on approximations due to the difficulty of estimating mutual information. We demonstrate that approximations made by existing methods are based on unrealistic assumptions. We formulate a more flex- ible and general class of assumptions based on variational distributions and use them to tractably generate lower bounds for mutual information. These bounds define a novel information-theoretic framework for feature selection, which we prove to be optimal under tree graphical models with proper choice of variational distributions. Our experiments demonstrate that the proposed method strongly outperforms existing information-theoretic feature selection approaches. 1 Introduction Feature selection is one of the fundamental problems in machine learning research [1, 2]. Many problems include a large number of features that are either irrelevant orredundant for the task at hand. In these cases, it is often advantageous to pick a smaller subset of features to avoid over-fitting, to speed up computation, or simply to improve the interpretability of the results. Feature selection approaches are usually categorized into three groups: wrapper ,embedded and filter [3, 4, 5]. The first two methods, wrapper andembedded , are considered classifier-dependent , i.e., the selection of features somehow depends on the classifier being used. Filter methods, on the other hand, are classifier-independent and define a scoring function between features and labels in the selection process. Because filter methods may be employed in conjunction with a wide variety of classifiers, it is im- portant that the scoring function of these methods is as general as possible. Since mutual information (MI) is a general measure of dependence with several unique properties [6], many MI-based scoring functions have been proposed as filter methods [7, 8, 9, 10, 11, 12]; see [5] for an exhaustive list. Owing to the difficulty of estimating mutual information in high dimensions, most existing MI-based feature selection methods are based on various low-order approximations for mutual information. While those approximations have been successful in certain applications, they are heuristic in nature and lack theoretical guarantees. In fact, as we demonstrate below (Sec. 2.2), a large family of approximate methods are based on two assumptions that are mutually inconsistent. To address the above shortcomings, in this paper we introduce a novel feature selection method based on variational lower bound on mutual information; a similar bound was previously studied within the Infomax learning framework [13]. We show that instead of maximizing the mutual infor- mation, which is intractable in high dimensions (hence the introduction of many heuristics), we can maximize a lower bound on the MI with the proper choice of tractable variational distributions. We use this lower bound to define an objective function and derive a forward feature selection algorithm.arXiv:1606.02827v1 [stat.ML] 9 Jun 2016We provide a rigorous proof that the forward feature selection is optimal under tree graphical models by choosing an appropriate variational distribution. This is in contrast with previous information- theoretic feature selection methods, which lack any performance guarantees. We also conduct em- pirical validation on various datasets and demonstrate that the proposed approach outperforms state- of-the-art information-theoretic feature selection methods. In Sec. 2 we introduce general MI-based feature selection methods and discuss their limitations. Sec. 3 introduces the variational lower bound on mutual information and proposes two specific vari- ational distributions. In Sec. 4, we report results from our experiments, and compare the proposed approach with existing methods. 2 Information-Theoretic Feature Selection Background 2.1 Mutual Information-Based Feature Selection Consider a supervised learning scenario where x=fx1;x2;:::;xDgis aD-dimensional input fea- ture vector, and yis the output label. In filter methods, the mutual information-based feature selec- tion task is to select Tfeatures xS=fxf1;xf2;:::;xfTgsuch that the mutual information between xSandyis maximized. Formally, S= arg max SI(xS:y)s:t:jSj=T(1) whereI(
)denotes the mutual information [6]. Forward Sequential Feature Selection Maximizing the objective function in Eq. 1 is generally NP-hard. Many MI-based feature selection methods adopt a greedy method, where features are selected incrementally, one feature at a time. Let St1=fxf1;xf2;:::;xft1gbe the selected feature set after time step t1. According to the greedy method, the next feature ftat steptis selected such that ft= arg max i=2St1I(xSt1[i:y)(2) where xSt1[idenotes x’s projection into the feature space St1[i. As shown in [5], the mutual information term in Eq. 2 can be decomposed as: I(xSt1[i:y) =I(xSt1:y) +I(xi:yjxSt1) =I(xSt1:y) +I(xi:y)I(xi:xSt1) +I(xi:xSt1jy) =I(xSt1:y) +I(xi:y) (H(xSt1)H(xSt1jxi)) + (H(xSt1jy)H(xSt1jxi;y))(3) whereH(
)denotes the entropy [6]. Omitting the terms that do not depend on xiin Eq. 3, we can rewrite Eq. 2 as follows: ft= arg max i=2St1I(xi:y) +H(xSt1jxi)H(xSt1jxi;y)(4) The greedy learning algorithm has been analyzed in [14]. 2.2 Limitations of Previous MI-Based Feature Selection Methods Estimating high-dimensional information-theoretic quantities is a difficult task. Therefore most MI-based feature selection methods propose low-order approximation to H(xSt1jxi)and H(xSt1jxi;y)in Eq. 4. A general family of methods rely on the following approximations [5]: H(xSt1jxi)t1X k=1H(xfkjxi) H(xSt1jxi;y)t1X k=1H(xfkjxi;y)(5) The approximations in Eq. 5 become exact under the following two assumptions [5]: 2Assumption 1. (Feature Independence Assumption) p(xSt1jxi) =t1Q k=1p(xfkjxi) Assumption 2. (Class-Conditioned Independence Assumption) p(xSt1jxi;y) =t1Q k=1p(xfkjxi;y) Assumption 1 and Assumption 2 mean that the selected features are independent and class- conditionally independent, respectively, given the unselected feature xiunder consideration. Assumption 1 Assumption 2 Satisfying both Assumption 1 and Assumption 2 Figure 1: The first two graphical models show the assumptions of traditional MI-based feature selec- tion methods. The third graphical model shows a scenario when both Assumption 1 andAssumption 2are true. Dashed line indicates there may or may not be a correlation between two variables. We now demonstrate that the two assumptions cannot be valid simultaneously unless the data has a very specific (and unrealistic) structure. Indeed, consider the graphical models consistent with either assumption, as illustrated in Fig. 1. If Assumption 1 holds true, then xiis the only common cause of the previously selected features St1=fxf1;xf2;:::;xft1g, so that those features become independent when conditioned on xi. On the other hand, if Assumption 2 holds, then the features depend both on xiand class label y; therefore, generally speaking, distribution over those features does not factorize by solely conditioning on xi—there will be remnant dependencies due to y. Thus, ifAssumption 2 is true, then Assumption 1 cannot be true in general, unless the data is generated according to a very specific model shown in the rightmost model in Fig. 1. Note, however, that in this case, xibecomes the most important feature because I(xi:y)>I(xSt1:y); then we should have selected xiat the very first step, contradicting the feature selection process. As we mentioned above, most existing methods implicitly or explicitly adopt both assumptions or their stronger versions as shown in [5], including mutual information maximization (MIM) [15], joint mutual information (JMI) [8], conditional mutual information maximization (CMIM) [9], maximum relevance minimum redundancy (mRMR) [10], conditional infomax feature extraction (CIFE) [16], etc. Approaches based on global optimization of mutual information, such as quadratic programming feature selection ( QPFS ) [11] and state-of-the-art conditional mutual information- based spectral method ( SPECCMI ) [12], are derived from the previous greedy methods and there- fore also implicitly rely on those two assumptions. In the next section we address these issues by introducing a novel information-theoretic framework for feature selection. Instead of estimating mutual information and making mutually inconsistent assumptions, our framework formulates a tractable variational lower bound on mutual information, which allows a more flexible and general class of assumptions via appropriate choices of variational distributions. 3 Method 3.1 Variational Mutual Information Lower Bound Letp(x;y)be the joint distribution of input ( x) and output ( y) variables. Barber & Agkov [13] derived the following lower bound for mutual information I(x:y)by using the non-negativity of KL-divergence, i.e.,P xp(xjy) logp(xjy) q(xjy)0gives: I(x:y)H(x) +hlnq(xjy)ip(x;y) (6) where angled brackets represent averages and q(xjy)is an arbitrary variational distribution. This bound becomes exact if q(xjy)p(xjy). 3It is worthwhile to note that in the context of unsupervised representation learning, p(yjx)and q(xjy)can be viewed as an encoder and a decoder , respectively. In this case, yneeds to be learned by maximizing the lower bound in Eq. 6 by iteratively adjusting the parameters of the encoder and decoder, such as [13, 17]. 3.2 Variational Information Maximization for Feature Selection Naturally, in terms of information-theoretic feature selection, we could also try to optimize the variational lower bound in Eq. 6 by choosing a subset of features Sinx, such that, S= arg max Sn H(xS) +hlnq(xSjy)ip(xS;y)o (7) However, the H(xS)term in RHS of Eq. 7 is still intractable when xSis very high-dimensional. Nonetheless, by noticing that variable yis the class label, which is usually discrete, and hence H(y) is fixed and tractable, by symmetry we switch xandyin Eq. 6 and rewrite the lower bound as follows: I(x:y)H(y) +hlnq(yjx)ip(x;y) = lnq(yjx) p(y) p(x;y)(8) The equality in Eq. 8 is obtained by noticing that H(y) =hlnp(y)ip(y). By using Eq. 8, the lower bound optimal subset Sofxbecomes: S= arg max S( lnq(yjxS) p(y) p(xS;y)) (9) 3.2.1 Choice of Variational Distribution q(yjxS)in Eq. 9 can be anydistribution as long as it is normalized. We need to choose q(yjxS)to be as general as possible while still keeping the term hlnq(yjxS)ip(xS;y)tractable in Eq. 9. As a result, we set q(yjxS)as q(yjxS) =q(xS;y) q(xS)=q(xSjy)p(y)P y0q(xSjy0)p(y0) (10) We can verify that Eq. 10 is normalized even if q(xSjy)is not normalized. If we further denote, q(xS) =X y0q(xSjy0)p(y0)(11) then by combining Eqs. 9, 10, we get, I(xS:y) lnq(xSjy) q(xS) p(xS;y)ILB(xS:y) (12) Auto-Regressive Decomposition. Now thatq(yjxS)is defined, all we need to do is model q(xSjy)under Eq. 10, and q(xS)is easy to compute based on q(xSjy). Here we decompose q(xSjy)as an auto-regressive distribution assuming Tfeatures inS: q(xSjy) =q(xf1jy)TY t=2q(xftjxf<t;y) (13) where xf<tdenotesfxf1;xf2;:::;xft1g. The graphical model in Fig. 2 demonstrates this decom- 4Figure 2: Auto-regressive decomposition for q(xSjy) position. The main advantage of this model is that it is well-suited for the forward feature selection procedure where one feature is selected at a time (which we will explain in Sec. 3.2.3). And if q(xftjxf<t;y)is tractable, then so is the whole distribution q(xSjy). Therefore, we would find tractableQ-Distributions over q(xftjxf<t;y). Below we illustrate two such Q-distributions. Naive Bayes Q-distribution. An natural idea would be to assume xtis independent of other variables given y, i.e., q(xftjxf<t;y) =p(xftjy) (14) Then the variational distribution q(yjxS)can be written based on Eqs. 10 and 14 as follows: q(yjxS) =p(y)Q j2Sp(xjjy) P y0p(y0)Q j2Sp(xjjy0)(15) And we also have the following theorem: Theorem 3.1 (Exact Naive Bayes) .Under Eq. 15, the lower bound in Eq. 8 becomes exact if and only if data is generated by a Naive Bayes model, i.e., p(x;y) =p(y)Q ip(xijy). The proof for Theorem 3.1 becomes obvious by using the mutual information definition. Note that the most-cited MI-based feature selection method mRMR [10] also assumes conditional indepen- dence given the class label yas shown in [5, 18, 19], but they make additional stronger independence assumptions among only feature variables. PairwiseQ-distribution. We now consider an alternative approach that is more general than the Naive Bayes distribution: q(xftjxf<t;y) = t1Y i=1p(xftjxfi;y)! 1 t1 (16) In Eq. 16, we assume q(xftjxf<t;y)to be the geometric mean of conditional distributions q(xftjxfi;y). This assumption is tractable as well as reasonable because if the data is gener- ated by a Naive Bayes model, the lower bound in Eq. 8 also becomes exact using Eq. 16 due to p(xftjxfi;y)p(xftjy)in that case. 3.2.2 Estimating Lower Bound From Data Assuming either Naive Bayes Q-distribution or Pairwise Q-distribution, it is convenient to estimate q(xSjy)andq(xS)in Eq. 12 by using plug-in probability estimators for discrete data or one/two- dimensional density estimator for continuous data. We also use the sample mean to approximate the expectation term in Eq. 12. Our final estimator for ILB(xS:y)is written as follows: bILB(xS:y) =1 NX x(k);y(k)lnbq x(k) Sjy(k) bq x(k) S (17) where x(k);y(k) are samples from data, and bq(
)denotes the estimate for q(
). 53.2.3 Variational Forward Feature Selection Under Auto-Regressive Decomposition After defining q(yjxS)in Eq. 10 and auto-regressive decomposition of q(xSjy)in Eq. 14, we are able to do the forward feature selection previously described in Eq. 2, but replace the mutual infor- mation with its lower bound bILB. Recall that St1is the set of selected features after step t1, then the feature ftwill be selected at step tsuch that ft= arg max i=2St1bILB(xSt1[i:y) (18) wherebILB(xSt1[i:y)can be obtained from bILB(xSt1:y)recursively by auto-regressive de- composition q(xSt1[ijy) =q(xSt1jy)q(xijxSt1;y)whereq(xSt1jy)is stored at step t1. This forward feature selection can be done under auto-regressive decomposition in Eqs. 10 and 13 foranyQ-distribution. However, calculating q(xijxSt;y)may vary according to different Q- distributions. We can verify that it is easy to get q(xijxSt;y)recursively from q(xijxSt1;y)under Naive Bayes or Pairwise Q-distribution. We call our algorithm under these two Q-distributions VMI naive andVMI pairwise respectively. It is worthwhile noting that the lower bound does not always increase at each step. A decrease in lower bound at step tindicates that the Q-distribution would approximate the underlying distribu- tion worse than it did at previous step t1. In this case, the algorithm would re-maximize the lower bound from zero with only the remaining unselected features. We summarize the concrete implementation of our algorithms in supplementary Sec. A. Time Complexity. Although our algorithm needs to calculate the distributions at each step, we only need to calculate the probability value at each sample point. For both VMI naive and VMI pairwise , the total computational complexity is O(NDT )assumingNas number of samples, Das total number of features, Tas number of final selected features. The detailed time analysis is left for the supplementary Sec. A. As shown in Table 1, our methods VMI naive andVMI pairwise have the same time complexity as mRMR [10], while state-of-the-art global optimization method SPECCMI [12] is required to precompute the pairwise mutual information matrix, which gives an time complexity of O(ND2). Table 1: Time complexity in number of features D, selected number of features d, and number of samplesN Method mRMRVMI naiveVMI pairwiseSPECCMI Complexity O(NDT )O(NDT )O(NDT )O(ND2) Optimality Under Tree Graphical Models. Although our method VMI naive assumes a Naive Bayes model, we can prove that this method is still optimal if the data is generated according to tree graphical models. Indeed, both of our methods, VMI naive andVMI pairwise , will always prioritize the first layer features, as shown in Fig. 3. This optimality is summarized in Theorem B.1 in supplementary Sec. B. 4 Experiments We begin with the experiments on a synthetic model according to the tree structure illustrated in the left part of Fig. 3. The detailed data generating process is shown in supplementary section D. The root node Yis a binary variable, while other variables are continuous. We use VMI naive to optimize the lower bound ILB(x:y).5000 samples are used to generate the synthethic data, and variational Q-distributions are estimated by kernel density estimator. We can see from the plot in the right part of Fig. 3 that our algorithm, VMI naive , selects x1,x2,x3as the first three features, although x2andx3are only weakly correlated with y. If we continue to add deeper level featuresfx4;:::;x9g, the lower bound will decrease. For comparison, we also illustrate the mutual information between each single feature xiandyin Table 2. We can see from Table 2 that it would choose x1,x4andx5as the top three features by using the maximum relevance criteria [15]. 60 2 4 6 8 Step t0.000.050.100.150.20Mutual Informationx1x2x3x9x8x6 x4 x7 x5 Ground Truth: I(xSt:y) Lower Bound: /hatwideILB(xSt:y)Figure 3: (Left) This is the generative model used for synthetic experiments. Edge thickness repre- sents the relationship strength. (Right) Optimizing the lower bound by VMI naive . Variables under the blue line denote the features selected at each step. Dotted blues line shows the decreasing lower bound if adding more features. Ground-truth mutual information is obtained using N= 100;000 samples. feature i x1 x2 x3 x4 x5 x6 x7 x8 x9 I(xi:y)0.111 0.052 0.022 0.058 0.058 0.025 0.029 0.012 0.013 Table 2: Mutual information between label yand each feature xifor Fig. 3.I(xi:y)is estimated using N=100,000 samples. Top three variables with highest mutual information are highlighted in bold. 4.1 Real-World Data We compare our algorithms VMI naive andVMI pairwise with other popular information-theoretic feature selection methods, including mRMR [10], JMI [8], MIM [15], CMIM [9], CIFE [16], and SPECCMI [12]. We use 17 well-known datasets in previous feature selection studies [5, 12] (all data are discretized). The dataset summaries are illustrated in supplementary Sec. C. We use the average cross-validation error rate on the range of 10 to 100 features to compare different algorithms under the same setting as [12]. 10-fold cross-validation is employed for datasets with number of samples N100and leave-one-out cross-validation otherwise. The 3-Nearest-Neighbor classifier is used for Gisette and Madelon, following [5]. While for the remaining datasets, the classifier is chosen to be Linear SVM, following [11, 12]. The experimental results can be seen in Table 31. The entries withandindicate the best perfor- mance and the second best performance respectively (in terms of average error rate). We also use the paired t-test at 5% significant level to test the hypothesis that VMI naive orVMI pairwise performs significantly better than other methods, or vice visa. Overall, we find that both of our methods, VMInaive andVMIpairwise , strongly outperform other methods, indicating our variational feature selection framework is a promising addition to the current literature of information-theoretic feature selection. 10 20 30 40 50 60 70 80 90 100 Number of selected features0.10.20.30.40.50.6Average cross validate errorSEMEION mRMR JMI MIM CMIM CIFE SPECCMI VMInaive VMIpairwise 10 20 30 40 50 60 70 80 90 100 Number of selected features0.030.040.050.060.070.080.090.10Average cross validate errorGISETTE mRMR JMI MIM CMIM CIFE SPECCMI VMInaive VMIpairwise Figure 4: Number of selected features versus average cross- validation error in datasets Semeion and Gisette. 1we omit the results for MIM andCIFE due to space limitations, the complete results are shown in the supplementary Sec. C. 7Table 3: Average cross-validation error rate comparison of VMI against other methods. The last two lines indicate win(W)/tie(T)/loss(L) for VMI naive andVMI pairwise respectively. Dataset mRMR JMI CMIM SPECCMIVMI naiveVMI pairwise Lung 10.9(4.7)11.6(4.7) 11.4(3.0) 11.6(5.6) 7.4(3.6)14.5(6.0) Colon 19.7(2.6) 17.3(3.0) 18.4(2.6) 16.1(2.0) 11.2(2.7)11.9(1.7) Leukemia 0.4 (0.7) 1.4(1.2) 1.1(2.0) 1.8(1.3) 0.0(0.1)0.2(0.5) Lymphoma 5.6 (2.8) 6.6(2.2) 8.6(3.3) 12.0(6.6) 3.7(1.9)5.2(3.1) Splice 13.6(0.4)13.7(0.5)14.7(0.3) 13.7(0.5)13.7(0.5)13.7(0.5) Landsat 19.5 (1.2) 18.9(1.0) 19.1(1.1) 21.0(3.5) 18.8(0.8)18.8(1.0) Waveform 15.9(0.5)15.9(0.5)16.0(0.7) 15.9(0.6)15.9(0.6)15.9(0.5) KrVsKp 5.1(0.7)5.2(0.6) 5.3(0.5) 5.1(0.6)5.3(0.5) 5.1(0.7) Ionosphere 12.8 (0.9) 16.6(1.6) 13.1(0.8) 16.8(1.6) 12.7(1.9)12.0(1.0) Semeion 23.4 (6.5) 24.8(7.6) 16.3(4.4) 26.0(9.3) 14.0(4.0)14.5(3.9) Multifeat. 4.0 (1.6) 4.0(1.6) 3.6(1.2) 4.8(3.0) 3.0(1.1)3.5(1.1) Optdigits 7.6 (3.3) 7.6(3.2) 7.5(3.4)9.2(6.0) 7.2(2.5)7.6(3.6) Musk2 12.4(0.7)12.8(0.7) 13.0(1.0) 15.1(1.8) 12.8(0.6) 12.6(0.5) Spambase 6.9 (0.7) 7.0(0.8) 6.8(0.7)9.0(2.3) 6.6(0.3)6.6(0.3) Promoter 21.5 (2.8) 22.4(4.0) 22.1(2.9) 24.0(3.7) 21.2(3.9)20.4(3.1) Gisette 5.5(0.9) 5.9(0.7) 5.1(1.3) 7.1(1.3) 4.8(0.9)4.2(0.8) Madelon 30.8 (3.8) 15.3(2.6)17.4(2.6) 15.9(2.5)16.7(2.7) 16.6(2.9) #W1=T1=L1: 11/4/2 10/6/1 10/7/0 13/2/2 #W2=T2=L2: 9/6/2 9/6/2 13/3/1 12/3/2 We also plot the average cross- validation error with respect to number of selected features. Fig. 4 shows the two most distinguishable data sets, Semeion and Gisette. We can see that both of our methods,VMI Naive andVMI Pairwise , have lower error rates in these two data sets. 5 Related Work There has been a significant amount of work on information-theoretic feature selection in the past twenty years: [5, 7, 8, 9, 10, 15, 11, 12, 20], to name a few. Most of these methods are based on combinations of so-called relevant, redundant andcomplimentary information. Such combinations representing low-order approximations of mutual information are derived from two assumptions, and it has proved unrealistic to expect both assumptions to be true. Inspired by group testing [21], more scalable feature selection methods have been developed, but this method also requires the calculation of high-dimensional mutual information as a basic scoring function. Estimating mutual information from data requires an large number of observations—especially when the dimensionality is high. The proposed variational lower bound can be viewed as a way of estimating mutual information between a high-dimensional continuous variable and a discrete variable. Only a few examples exist in literature [22] under this setting. We hope our method will shed light on new ways to estimate mutual information, similar to estimating divergences in [23]. 6 Conclusion Feature selection has been a significant endeavor over the past decade. Mutual information gives a general basis for quantifying the informativeness of features. Despite the clarity of mutual in- formation, estimating it can be difficult. While a large number of information-theoretic methods exist, they are rather limited and rely on mutually inconsistent assumptions about underlying data distributions. We introduced a unifying variational mutual information lower bound to address these issues. We showed that by auto-regressive decomposition, feature selection can be done in a forward manner by progressively maximizing the lower bound. We also presented two concrete methods us- ing Naive Bayes and Pairwise Q-distributions, which strongly outperform the existing methods. VMI naive only assumes a Naive Bayes model, but even this simple model outperforms the existing information-theoretic methods, indicating the effectiveness of our variational information maximiza- tion framework. We hope that our framework will inspire new mathematically rigorous algorithms for information-theoretic feature selection, such as optimizing the variational lower bound globally and developing more powerful variational approaches for capturing complex dependencies. 8References [1] Manoranjan Dash and Huan Liu. Feature selection for classification. Intelligent data analysis , 1(3):131– 156, 1997. [2] Huan Liu and Hiroshi Motoda. Feature selection for knowledge discovery and data mining , volume 454. Springer Science & Business Media, 2012. [3] Ron Kohavi and George H John. Wrappers for feature subset selection. Artificial intelligence , 97(1):273– 324, 1997. [4] Isabelle Guyon and Andr ´e Elisseeff. An introduction to variable and feature selection. The Journal of Machine Learning Research , 3:1157–1182, 2003. [5] Gavin Brown, Adam Pocock, Ming-Jie Zhao, and Mikel Luj ´an. Conditional likelihood maximisation: a unifying framework for information theoretic feature selection. The Journal of Machine Learning Re- search , 13(1):27–66, 2012. [6] Thomas M Cover and Joy A Thomas. Elements of information theory . John Wiley & Sons, 2012. [7] Roberto Battiti. Using mutual information for selecting features in supervised neural net learning. Neural Networks, IEEE Transactions on , 5(4):537–550, 1994. [8] Howard Hua Yang and John E Moody. Data visualization and feature selection: New algorithms for nongaussian data. In NIPS , volume 99, pages 687–693. Citeseer, 1999. [9] Franc ¸ois Fleuret. Fast binary feature selection with conditional mutual information. The Journal of Machine Learning Research , 5:1531–1555, 2004. [10] Hanchuan Peng, Fuhui Long, and Chris Ding. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. Pattern Analysis and Machine Intelligence, IEEE Transactions on , 27(8):1226–1238, 2005. [11] Irene Rodriguez-Lujan, Ramon Huerta, Charles Elkan, and Carlos Santa Cruz. Quadratic programming feature selection. The Journal of Machine Learning Research , 11:1491–1516, 2010. [12] Xuan Vinh Nguyen, Jeffrey Chan, Simone Romano, and James Bailey. Effective global approaches for mutual information based feature selection. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining , pages 512–521. ACM, 2014. [13] David Barber and Felix Agakov. The im algorithm: a variational approach to information maximiza- tion. In Advances in Neural Information Processing Systems 16: Proceedings of the 2003 Conference , volume 16, page 201. MIT Press, 2004. [14] Abhimanyu Das and David Kempe. Submodular meets spectral: Greedy algorithms for subset selection, sparse approximation and dictionary selection. In Proceedings of the 28th International Conference on Machine Learning (ICML-11) , pages 1057–1064, 2011. [15] David D Lewis. Feature selection and feature extraction for text categorization. In Proceedings of the workshop on Speech and Natural Language , pages 212–217. Association for Computational Linguistics, 1992. [16] Dahua Lin and Xiaoou Tang. Conditional infomax learning: an integrated framework for feature extrac- tion and fusion. In Computer Vision–ECCV 2006 , pages 68–82. Springer, 2006. [17] Shakir Mohamed and Danilo Jimenez Rezende. Variational information maximisation for intrinsically motivated reinforcement learning. In Advances in Neural Information Processing Systems , pages 2116– 2124, 2015. [18] Kiran S Balagani and Vir V Phoha. On the feature selection criterion based on an approximation of multidimensional mutual information. IEEE Transactions on Pattern Analysis & Machine Intelligence , (7):1342–1343, 2010. [19] Nguyen Xuan Vinh, Shuo Zhou, Jeffrey Chan, and James Bailey. Can high-order dependencies improve mutual information based feature selection? Pattern Recognition , 2015. [20] Hongrong Cheng, Zhiguang Qin, Chaosheng Feng, Yong Wang, and Fagen Li. Conditional mutual information-based feature selection analyzing for synergy and redundancy. ETRI Journal , 33(2):210– 218, 2011. [21] Yingbo Zhou, Utkarsh Porwal, Ce Zhang, Hung Q Ngo, Long Nguyen, Christopher R ´e, and Venu Govin- daraju. Parallel feature selection inspired by group testing. In Advances in Neural Information Processing Systems , pages 3554–3562, 2014. [22] Brian C Ross. Mutual information between discrete and continuous data sets. PloS one , 9(2):e87357, 2014. [23] XuanLong Nguyen, Martin J Wainwright, and Michael I Jordan. Estimating divergence functionals and the likelihood ratio by convex risk minimization. Information Theory, IEEE Transactions on , 56(11):5847–5861, 2010. [24] Shuyang Gao. Variational feature selection code. http://github.com/BiuBiuBiLL/ InfoFeatureSelection . [25] Chris Ding and Hanchuan Peng. Minimum redundancy feature selection from microarray gene expression data. Journal of bioinformatics and computational biology , 3(02):185–205, 2005. [26] Kevin Bache and Moshe Lichman. Uci machine learning repository, 2013. 9Supplementary Material for “Variational Information Maximization for Feature Selection” A Detailed Algorithm for Variational Forward Feature Selection We describe the detailed algorithm for our approach. We also provide open source code implement- ingVMI naive andVMI pairwise [24]. Concretely, let us suppose class label yis discrete and has Ldifferent valuesfy1;y2;:::;y Lg; then we define the distribution q(xStjy)vectorQ(k) tof sizeLfor each sample x(k);y(k)
at stept: Q(k) t=h bq x(k) Stjy=y1 ;:::;bq x(k) Stjy=yLiT(19) where x(k) Stdenotes the sample x(k)projects onto the xStfeature space. Also, We further denote Y of size L1as the distribution vector of yas follows: Y= [bp(y=y1);bp(y=y2);:::;bp(y=yL)]T(20) Then we are able to rewrite q(xSt1)andq(xSt1jy)in terms ofQ(k) t1;Yand substitute them into bILB(xSt1:y). To illustrate, at step t1we have, bILB(xSt1:y) =1 NX x(k);y(k)log p x(k) St1jy=y(k) 1 NX klog YTQ(k) t1 (21) To select a feature iat stept, let us define the conditional distribution vector C(k) i;t1for each feature i =2St1and each sample x(k);y(k)
, i.e., C(k) i;t1=h q x(k) ijx(k) St1;y=y1 ;:::;q x(k) ijx(k) St1;y=yLiT(22) At stept, we useC(k) i;t1andQ(k) t1previously stored and get, bILB(xSt1[i:y) =1 NX x(k);y(k)log p x(k) St1jy=y(k) p x(k) ijx(k) St1;y=y(k) 1 NX klog YTdiag Q(k) t1 C(k) i;t1 (23) We summarize our detailed implementation in Algorithm 1. UpdatingQ(k) tandC(k) i;tin Algorithm 1 may vary according to different Q-distributions. But we can verify that under Naive Bayes Q-distribution or Pairwise Q-distribution, Q(k) tandC(k) i;tcan be ob- tained recursively from Q(k) t1andC(k) i;t1by noticing that q(xijxSt;y) =p(xijy)for Naive Bayes Q-distribution and q(xijxSt;y) = p(xijxft;y)q(xijxSt1;y)t1t for Pairwise Q-distribution. Let us denote Nas number of samples, Das total number of features, Tas number of selected features and Las number of distinct values in class variable y. The computational complex- ity of Algorithm 1 involves calculating the lower bound for each feature iat every step which is O(NDL ); updatingC(k) i;twould costO(NDL )for pairwise Q-distribution and O(1)for Naive BayesQ-distribution; updating Q(k) twould costO(NDL ). We need to select Tfeatures, therefore the time complexity is O(NDT )2. 2we ignore Lhere because the number of classes is usually much smaller. 10Algorithm 1 Variational Forward Feature Selection (VMI) Data: x(1);y(1)
; x(2);y(2)
;:::; x(N);y(N)
Input:T fnumber of features to select g Output:F ffinal selected feature set g F f?g;S0 f?g;t 1 InitializeQ(k) 0andC(k) i;0for any feature i; calculateY whilejFj<Tdo bILB(xSt1[i:y) fEq. 23 for each inot inFg ft arg max i=2St1bILB(xi[St1:y) ifbILB xSt1[ft:y
bILB(xSt1:y)then ClearS; Sett 1 else F F[ft St St1[ft UpdateQ(k) tandC(k) i;t t t+ 1 end if end while B Optimality under Tree Graphical Models Theorem B.1 (Optimal Feature Selection) .If data is generated according to tree graphical models, where the class label yis the root node, denote the child nodes set in the first layer as L1= fx1;x2;:::;xL1g, as shown in Fig. B.1. Then there must exist a step T > 0such that the following three conditions hold by using VMI naive orVMI pairwise : Condition I: The selected feature set STL 1. Condition II: ILB(xSt:y) =I(xSt:y)for1tT. Condition III: ILB(xST:y) =I(x:y). Figure B.1: Demonstration of tree graphical model, label yis the root node. Proof. We prove this theorem by induction. For tree graphical model when selecting the first layer features,VMI naive andVMI pairwise are mathematically equal, therefore we only prove VMI naive case andVMI pairwise follows the same proof. 111) At stept= 1, for each feature i, we have, ILB(xi:y) = lnq(xijy) q(xi) p(x;y) =* ln0 B@p(xijy)P y0p(y0)p(xijy0)1 CA+ p(x;y) = lnp(xijy) p(xi) p(x;y)=I(xi:y)(24) Thus, we are choosing a feature that has the maximum mutual information with yat the very first step. Based on the data processing inequality, we have I(xi:y)I(desc(xi) :y)for any xiin layer 1 where desc(xi)represents any descendant of xi. Thus, we always select features among the nodes of the first layer at step t= 1 without loss of generality. If node xjthat is not in the first layer is selected at step t= 1, denoteances (xj)asxj’s ancestor in layer 1, then I(xj:y) =I(ances (xj) :y)which means that the information is not lost from ances (xj)!xj. In this case, one can always switch ances (xj)with xjand let xjbe in the first layer, which does not conflict with the model assumption. Therefore, condition I and II are satisfied in step t= 1. 2) Assuming condition I and II are satisfied in step t, then we have the following argument in step t+ 1: We discuss the candidate nodes in three classes, and argue that nodes in Remaining-Layer 1 Class are always being selected. Redundant Class For any descendant desc(St)of selected feature set St, we have, I xSt[desc (St):y
=I(xSt:y) =ILB(xSt:y) (25) Eq. 25 comes from the fact that the desc(St)carries no additional information about yother than St. The second equality is by induction. Based on Eq. 12 and 25, we have, ILB xSt[desc (St):y
<I xSt[desc (St):y
=I(xSt:y)(26) We assume here that the LHS is strictly less than RHS in Eq. 26 without loss of generality. This is because if the equality holds, we have p(xStjy)p(desc (St)jy) =p(xt;desc (St)jy)due to Theorem 3.1. In this case, we can always rearrange desc(St)to the first layer, which does not conflict with the model assumption. Note that by combining Eqs. 25 and 26, we can also get ILB xSt[desc (St):y
<ILB(xSt:y) (27) Eq. 27 means that adding a feature in Redundant Class will actually decrease the value of lower boundILB. Remaining-Layer1 Class For any other unselected node jof the first layer, i.e., j2L 1nSt, we have I(xSt:y)I(xSt[j:y) =ILB(xSt[j:y) (28) The inequality in Eq. 28 is obvious which comes from the data processing inequality [6]. And the equality in Eq. 28 comes directly from Theorem 3.1. Descendants-of-Remaining-Layer1 Class For any node desc(j)that is the descendant of jwhere j2L 1nSt, we have, ILB xSt[desc (j):y
I xSt[desc (j):y
I xSt[desc (j):y
I(xSt[j:y)(29) The second inequality of Ineq. 29 also comes from data processing inequality. 12Combining Eqs. 26 and 28, we get, ILB xSt[desc (St):y
<ILB(xSt[j:y) (30) Combining Eqs. 28 and 29, we get, ILB xSt[desc (j):y
ILB(xSt[j:y) (31) Ineq. 30 essentially tells us the forward feature selection will always choose Remaining-Layer1 Class other than Redundant Class . Ineq. 31 is saying we are choosing Remaining-Layer1 Class other than Descendants-of-Remaining- Layer1 Class without loss of generality (for the equality concern, we can have the same argument in stept= 1). Considering Ineqs. 30 and 31, in step t+ 1, the algorithm chooses node jinRemaining-Layer1 Class , i.e.,j2L 1nSt. Therefore, condition I and II hold at step t+ 1. At stept+ 1, ifILB(xSt[j:y) =ILB(xSt:y)for anyj2L 1nSt, that means I(xSt[j:y) = I(xSt:y). Then we have, I(xSt:y) =I(xL1:y) =I(x:y) (32) The first equality in Eq. 32 holds because adding any jinL1nStwill not increase the mutual information. The second equality is due to the data processing inequality under tree graphical model assumption. Therefore, if ILB(xSt[j:y) =ILB(xSt:y)for anyj2L 1nSt, we setT=t. Thus by combin- ing condition II and Eq. 32, we have, ILB(xST:y) =I(xST:y) =I(x:y) (33) Then condition III holds. C Datasets and Results Table 4 summarizes the datasets used in the experiment. Table 5 shows the complete results. Table 4: Dataset summary. N: # samples, d: # features, L: # classes. Data N d L Source Lung 73 325 20 [25] Colon 62 2000 2 [25] Leukemia 72 7070 2 [25] Lymphoma 96 4026 9 [25] Splice 3175 60 3 [26] Landsat 6435 36 6 [26] Waveform 5000 40 3 [26] KrVsKp 3196 36 2 [26] Ionosphere 351 34 2 [26] Semeion 1593 256 10 [26] Multifeat. 2000 649 10 [26] Optdigits 3823 64 10 [26] Musk2 6598 166 2 [26] Spambase 4601 57 2 [26] Promoter 106 57 2 [26] Gisette 6000 5000 2 [4] Madelon 2000 500 2 [4] 13Dataset mRMR JMI MIM CMIM CIFE SPECCMIVMI naiveVMI pairwise Lung 10.9(4.7)11.6(4.7) 18.3(5.4) 11.4(3.0) 23.3(5.4) 11.6(5.6) 7.4(3.6)14.5(6.0) Colon 19.7(2.6) 17.3(3.0) 22.0(4.3) 18.4(2.6) 23.5(4.3) 16.1(2.0) 11.2(2.7)11.9(1.7) Leukemia 0.4 (0.7) 1.4(1.2) 2.5(1.1) 1.1(2.0) 4.9(1.9) 1.8(1.3) 0.0(0.1)0.2(0.5) Lymphoma 5.6 (2.8) 6.6(2.2) 13.0(6.4) 8.6(3.3) 35.6(4.3) 12.0(6.6) 3.7(1.9)5.2(3.1) Splice 13.6(0.4)13.7(0.5) 13.6(0.5)13.7(0.5) 14.7(0.3) 13.7(0.5) 13.7(0.5) 13.7(0.5) Landsat 19.5(1.2) 18.9(1.0) 22.0(3.8) 19.1(1.1) 19.7(1.7) 21.0(3.5) 18.8(0.8)18.8(1.0) Waveform 15.9(0.5)15.9(0.5)16.1(0.8) 16.0(0.7) 22.8(2.2) 15.9(0.6)15.9(0.6)15.9(0.5) KrVsKp 5.1(0.7) 5.2(0.6) 5.3(0.6) 5.3(0.5) 5.0(0.7)5.1(0.6)5.3(0.5) 5.1(0.7) Ionosphere 12.8 (0.9) 16.6(1.6) 13.3(0.9) 13.1(0.8) 16.1(1.6) 16.8(1.6) 12.7(1.9)12.0(1.0) Semeion 23.4(6.5) 24.8(7.6) 26.7(9.7) 16.3(4.4) 28.6(5.8) 26.0(9.3) 14.0(4.0)14.5(3.9) Multifeat. 4.0 (1.6) 4.0(1.6) 4.9(2.3) 3.6(1.2) 7.2(3.0) 4.8(3.0) 3.0(1.1)3.5(1.1) Optdigits 7.6 (3.3) 7.6(3.2) 7.9(3.9) 7.5(3.4)8.1(4.2) 9.2(6.0) 7.2(2.5)7.6(3.6) Musk2 12.4(0.7)12.8(0.7) 14.0(1.2) 13.0(1.0) 13.2(0.6) 15.1(1.8) 12.8(0.6) 12.6(0.5) Spambase 6.9 (0.7) 7.0(0.8) 7.3(0.9) 6.8(0.7)10.3(1.8) 9.0(2.3) 6.6(0.3)6.6(0.3) Promoter 21.5 (2.8) 22.4(4.0) 21.7(3.1) 22.1(2.9) 27.4(3.2) 24.0(3.7) 21.2(3.9)20.4(3.1) Gisette 5.5(0.9) 5.9(0.7) 7.2(1.2) 5.1(1.3) 6.5(0.8) 7.1(1.3) 4.8(0.9)4.2(0.8) Madelon 30.8(3.8) 15.3(2.6)16.8(2.7) 17.4(2.6) 15.1(2.7)15.9(2.5) 16.7(2.7) 16.6(2.9) #W1=T1=L1: 11/4/2 10/6/1 11/6/0 10/7/0 15/0/2 13/2/2 #W2=T2=L2: 9/6/2 9/6/2 15/2/0 13/3/1 15/1/1 12/3/2 Table 5: Average cross validation error rate comparison of VMI against other methods. The last two lines indicate win(W)/tie(T)/loss(L) for VMI naive andVMI pairwise respectively. 14D Generating Synthetic Data Here is a detailed generating process for synthetic tree graphical model data in the experiment. Draw yBernoulli (0:5) Draw x1Gaussian (= 1:0;=y) Draw x2Gaussian (= 1:0;=y=1:5) Draw x3Gaussian (= 1:0;=y=2:25) Draw x4Gaussian (= 1:0;=x1) Draw x5Gaussian (= 1:0;=x1) Draw x6Gaussian (= 1:0;=x2) Draw x7Gaussian (= 1:0;=x2) Draw x8Gaussian (= 1:0;=x3) Draw x9Gaussian (= 1:0;=x3) 15

You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you s... You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you select really corresponds to the excerpt: there will be details mentioned in the excerpt that should appear in the paper. If you read an abstract and it seems like it could be the paper we’re looking for, read the paper to make sure. Also: sometimes you’ll read a paper that cites the paper we’re looking for. In such cases, please go to the references in order to find the full name of the paper we’re looking for, and search for it, and then select it. One reason might be the training task itself: from [CITATION] we know that ImageNet can be solved to high accuracy using only local information. In other words, it might simply suffice to integrate evidence from many local texture features rather than going through the process of integrating and classifying global shapes You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you select really corresponds to the excerpt: there will be details mentioned in the excerpt that should appear in the paper. If you read an abstract and it seems like it could be the paper we’re looking for, read the paper to make sure. Also: sometimes you’ll read a paper that cites the paper we’re looking for. In such cases, please go to the references in order to find the full name of the paper we’re looking for, and search for it, and then select it. One reason might be the training task itself: from [CITATION] we know that ImageNet can be solved to high accuracy using only local information. In other words, it might simply suffice to integrate evidence from many local texture features rather than going through the process of integrating and classifying global shapes

- Paper ID: bcd883ef8faa9a81f5c14da97669df1dd2e8f1b8 Title: Local directional relation pattern for unconstrained and robust face retrieva... - Paper ID: bcd883ef8faa9a81f5c14da97669df1dd2e8f1b8 Title: Local directional relation pattern for unconstrained and robust face retrieval Abstract: None Citation Count: 40 - Paper ID: b49064ba4ef3a8b0a9479e1f927c9b9e3cbdbfd2 Title: Joint Image Classification and Annotation Prediction Using Iterative Learning on Local Neighbourhood Abstract: Image annotation (tag) and classification play a critical role in many computer vision applications, such as image retrieval, scene understanding, scene description etc. While, databases such as ImageNet have high quality labels for images, in real world, a large number of images have missing labels or tags that completely describe the contents of an image. To solve this problem, in this paper, we work on the hypothesis that class and tag information are correlated and propose a joint optimization for image classification and annotation. We construct a unified cost function to learn the class scoring vectors as well as tag scoring v... - Paper ID: bcd883ef8faa9a81f5c14da97669df1dd2e8f1b8 Title: Local directional relation pattern for unconstrained and robust face retrieval Abstract: None Citation Count: 40 - Paper ID: b49064ba4ef3a8b0a9479e1f927c9b9e3cbdbfd2 Title: Joint Image Classification and Annotation Prediction Using Iterative Learning on Local Neighbourhood Abstract: Image annotation (tag) and classification play a critical role in many computer vision applications, such as image retrieval, scene understanding, scene description etc. While, databases such as ImageNet have high quality labels for images, in real world, a large number of images have missing labels or tags that completely describe the contents of an image. To solve this problem, in this paper, we work on the hypothesis that class and tag information are correlated and propose a joint optimization for image classification and annotation. We construct a unified cost function to learn the class scoring vectors as well as tag scoring vectors. The proposed approach achieves state-of-the-art results on benchmark datasets for joint tag prediction and classification. Citation Count: 0 - Paper ID: d62badc3f04a7919c23bf44a4ab5d7d765d2adbe Title: Information poverty-Connecting every child to opportunity Abstract: For a child to be nutritionally healthy, 1,500 calories are required each day. Can similar measures be defined with respect to information? If so, how many kilobytes and of what type does a child need to be "information healthy" and have equitable access to opportunity and choice? UNICEF is creating an open source platform to measure a child’s access to information and use this to build sustainable infrastructures and programmes that connect every child to opportunity and employment [1]. Here we explore ways Artificial Intelligence can help to connect every child in the world to the information they require to prepare them for the future. Being information poor The 21st century has seen a world transformed by connectivity. At a global scale connectivity has reshaped the lives of many in what has been called the information revolution. News, social connections, jobs, education, health services, and opportunities and choice in general have all adapted, and continue to change, to leverage the power and possibilities of this new age. This change, however, comes with challenges. Some of them have been extensively studied like issues of information overload [2], information bubbles [3], and the spread of fake news [4], while other, equally important topics, have been insufficiently studied. Underlying all of the Sustainable Development Goals (SDG), as well, is a thread of information and opportunity [5]. We will not be able to fully achieve these ambitious benchmarks for change if certain populations, particularly the young, are disconnected or ill-prepared for the future. A way of describing this sub-strata of the SDGs is through the lens of information poverty. Information is a driving force of opportunity and choice [6]. What does it mean to be information poor? This question is a highly complex one encompassing multiple dimensions of deprivation. Information poverty can limit many types of opportunity, from access to job skills, financing, creative and social networks, and the ability to develop the skills necessary for global citizenship [7]. Due to the complexity of this problem new sources of data about both information access and young peoples’ needs, along with AI and machine learning are required to help us understand information and infrastructure gaps of the most vulnerable. UNICEF, and partners, can use new models and datadriven approaches to work with governments to ensure provision of, and access to, relevant and necessary global digital public goods. Mapping the physical landscape In order to connect young people to opportunity and employment, we first need to understand where they live and what the infrastructure gaps around them are: Do children have access to schools and 32nd Conference on Neural Information Processing Systems (NIPS 2018), Montréal, Canada. education? Are they part of the information society? To answer these questions, UNICEF is mapping every school in the world and measuring their connectivity in real-time [8]. AI has been used to identify patterns in satellite imagery [9–11]. Nevertheless, schools, especially in rural areas of low and middle income countries present an additional challenge, since existing pre-trained neural networks such as ImageNet or ResNet [12, 13] are biased towards imagery and landscapes for high income regions. Schools are central for UNICEF’s work as they act as hubs within communities. Thus, UNICEF is working with governments to create a comprehensive school data corpus that accounts for regions that are not traditionally accounted. UNICEF is then using this data corpus to work with partners like UC San Diego and Development Seed to explore the use of high resolution satellite imagery and Deep Learning techniques to detect schools [14]. The initial results for the country of Liberia show great promise (76% overall accuracy) and demonstrate the feasibility of using AI to detect schools. This is, however, just a pilot project, and considerable work is required to put this into practice to be able to detect schools globally across a variation of contexts (urban, rural, regional). Similarly, remote sensing and deep learning techniques have shown promise in mapping infrastructures, including: roads [15], water bodies [16], urban/suburban structures [17] and power lines [18]. Can other sources of data such as mobility patterns, social media posts, or other data be used to fill in our knowledge gaps and if so how can we further the use of AI in this task? Having the base layer of school location and school connectivity allows us to start addressing infrastructure gaps and bring connectivity, information and opportunity to disconnected communities. However, how do we measure, at scale, whether schools are connected to the internet? Can AI help us do this and optimize its delivery? Mapping the digital landscape Complementary to addressing infrastructure gaps and connecting every school to the Internet, we need to make sure that children have access to the right content (i.e. in their own language, locally produced, and relevant to their context). Previous work has shown a correlation between the development of network infrastructure and the creation of local content [19]. Yet, the distribution of information and content across different geographies and languages still remains uneven [20]. While Wikipedia is a shining beacon of information it is a glaring example of how information is segregated, with different language editions varying dramatically in how comprehensive they are [21]. As a result, a majority of languages contain only a small fraction of the sum of information that exists for the English version. Recent advances in AI and Natural Language Processing (NLP) can aid us in solving and detecting these information gaps by analyzing and measuring content availability. Moreover, not all content is equal. As such, we value content that deals with local community higher than yellow press content, for example. NLP techniques can aid us in distilling and distinguishing content that is relevant to a specific context and allow us to develop mechanisms that can quantify content gaps. Finally, we can build human-in-the-loop solutions that combine the power of AI and human expertise to fill these gaps. Citation Count: 0 - Paper ID: de3760d7a8d53ceccd7e9c2181f72a3b0d6350a8 Title: Real-time multiple people tracking for automatic group-behavior evaluation in delivery simulation training Abstract: None Citation Count: 13 - Paper ID: 2050106ac2d0ba024c01538e30d62f263a33c438 Title: Research on the Fine-grained Plant Image Classification Abstract: The similarity between different subcategories and scarce training data due to the difficulties of Fine-grained recognition. Even in the same subcategories, there can be some differences due to the distinct color and pose of objects. We propose some models for fine-grained plant recognition by taking advantage of deep Convolutional Neural Network (CNN) and traditional feature based methods including SIFT [1], Bag of Word (BoW) [2]. We evaluate our method on Oxford 102 Flowers dataset [3], our results show that the CNN method achieves higher accuracy than the traditional feature based methods. Our results demonstrates state-of-the-art performances on the Oxford 102 Flowers with 88.40% (Acc.). Introduction Object recognition is one of the major focuses of research in computer vision. Most of existing recognition tasks are on basic-level: distinguishing between table, human, computer, car and so on. Categories differ greatly from each other on basic-level recognition. On the contrary, fine-grained recognition concentrates on differences between subcategory (breeds, species or product models), for example, recognition of different species of birds or species of flowers, which means similarities existing across categories and subtle differences needed to be found. Scale-invariant feature transform (SIFT) is an algorithm for local features detection and description. SIFT and its variants are frequently used in image matching and image retrieval to extract features. Since Sivic et al. [2] introduced the BoW method from natural language processing to computer vision and achieved great success on many public datasets, including 15-Scenes [4], Caltech-256 [5], PASCAL VOC [6] etc. CNN first was popularized by LeCun [7] to use in digit recognition, but fell out of fashion because of the requirement for strong computing power and large amounts of training data. With the development of parallel computing and the construction of large image databases, CNN goes to front stage again and achieves high success in many computer vision tasks. For instance, Krizhevsky et al. [8] achieved an impressive result using a CNN in ILSVRC2012 [9] with two GPUs to accelerate the computation of CNN parameters. Inspired by Krizhevsky et al., many groups proposed CNN architectures to solve the classification problems. In order to get a better performance, many CNNs ([10] [11] [12]) are first pre-trained on a large image set, ImageNet [9] for example, followed by domain-specific fine-tuning. Girshick et al. [10] proposed a model applied CNN to bottom-up region proposals and generalized the CNN classification results on ImageNet to Pascal VOC. N Zhang et al. [12] fine-tuned the ImageNet pre-trained CNN for the 200-way bird classify using the ground truth bounding box crops of the original images. In recent years, a variety of methods about find-grained classification have been proposed. We divide these methods into two parts. One is traditional feature based methods, usually using some methods to extract hand-made features and then using a classifier for classification. Another is CNN based methods, usually using a deep convolutional neural network to extract features and obtain the classification result automatically. In this paper, we propose our methods in both traditional hand-made features based and CNN. We combine the SIFT and BoW for image classification. Then we use CNN for image classification to compare to the method mentioned before. Our results show that CNN method can achieve higher 4th International Conference on Machinery, Materials and Information Technology Applications (ICMMITA 2016) Copyright © 2017, the Authors. Published by Atlantis Press. This is an open access article under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/). Advances in Computer Science Research, volume 71 Citation Count: 1

- Paper ID: 29dc9cf7005047a21d72bfd26ab35bd05c97e2e4 Title: Assessing Texture Features to Classify Coastal Wetland Vegetation from High S... - Paper ID: 29dc9cf7005047a21d72bfd26ab35bd05c97e2e4 Title: Assessing Texture Features to Classify Coastal Wetland Vegetation from High Spatial Resolution Imagery Using Completed Local Binary Patterns (CLBP) Abstract: Coastal wetland vegetation is a vital component that plays an important role in environmental protection and the maintenance of the ecological balance. As such, the efficient classification of coastal wetland vegetation types is key to the preservation of wetlands. Based on its detailed spatial information, high spatial resolution imagery constitutes an important tool for extracting suitable texture features for improving the accuracy of classification. In this paper, a texture feature, Completed Local Binary Patterns (CLBP), which is highly suitable for face recognition, is presented and applied to vegetation classification using high spatial resolution Pleiades satellite imagery in the central zone of Yancheng National Natural Reservation (YNNR) in Jiangsu, China.... - Paper ID: 29dc9cf7005047a21d72bfd26ab35bd05c97e2e4 Title: Assessing Texture Features to Classify Coastal Wetland Vegetation from High Spatial Resolution Imagery Using Completed Local Binary Patterns (CLBP) Abstract: Coastal wetland vegetation is a vital component that plays an important role in environmental protection and the maintenance of the ecological balance. As such, the efficient classification of coastal wetland vegetation types is key to the preservation of wetlands. Based on its detailed spatial information, high spatial resolution imagery constitutes an important tool for extracting suitable texture features for improving the accuracy of classification. In this paper, a texture feature, Completed Local Binary Patterns (CLBP), which is highly suitable for face recognition, is presented and applied to vegetation classification using high spatial resolution Pleiades satellite imagery in the central zone of Yancheng National Natural Reservation (YNNR) in Jiangsu, China. To demonstrate the potential of CLBP texture features, Grey Level Co-occurrence Matrix (GLCM) texture features were used to compare the classification. Using spectral data alone and spectral data combined with texture features, the image was classified using a Support Vector Machine (SVM) based on vegetation types. The results show that CLBP and GLCM texture features yielded an accuracy 6.50% higher than that gained when using only spectral information for vegetation classification. However, CLBP showed greater improvement in terms of classification accuracy than GLCM for Spartina alterniflora. Furthermore, for the CLBP features, CLBP_magnitude (CLBP_m) was more effective than CLBP_sign (CLBP_s), CLBP_center (CLBP_c), and CLBP_s/m or CLBP_s/m/c. These findings suggest that the CLBP approach offers potential for vegetation classification in high spatial resolution images. Citation Count: 35 - Paper ID: 50f35f9f20428f1551f1d8fd54f27d9e080862e4 Title: Grapes Visual Segmentation for Harvesting Robots Using Local Texture Descriptors Abstract: None Citation Count: 13 - Paper ID: 10e58a4fd2a23840d50ff4c540ad6ad21a602f9e Title: Content-based image retrieval using local texture features in distributed environment Abstract: In this paper, we propose novel content-based image retrieval (CBIR) algorithms using Local Octa Patterns (LOtP), Local Hexadeca Patterns (LHdP) and Direction Encoded Local Binary Pattern (DELBP). LOtP and LHdP encode the relationship between center pixel and its neighbors based on the pixels’ direction obtained by considering the horizontal, vertical and diagonal pixels for derivative calculations. In DELBP, direction of a referenced pixel is determined by considering every neighboring pixel for derivative calculations which results in 256 directions. For this resultant direction encoded image, we have obtained LBP which is considered as feature vector. The proposed method’s performance is compared to that of Local Tetra Patterns (LTrP) using benchmark image databases viz., Corel 1000 (DB1) and Brodatz textures (DB2). Performance analysis shows that LOtP improves the average precision from 59.31% to 64.36% on DB1, and from 83.24% to 85.95% on DB2, LHdP improves it to 65.82% on DB1 and to 87.49% on DB2 and DELBP improves it to 60.35% on DB1 and to 86.12% on DB2 as compared to that of LTrP. Also, DELBP reduces the feature vector length by 66.62% as compared to that of LTrP. To reduce the retrieval time, the proposed algorithms are implemented on a Hadoop cluster consisting of 116 nodes and tested using Corel 10K (DB3), Mirflickr 100,000 (DB4) and ImageNet 511,380 (DB5) databases. Citation Count: 10 - Paper ID: 10d7734ccfd6061c13d2f50f7eea5a6162479c36 Title: Single Classifier-Based Passive System for Source Printer Classification Using Local Texture Features Abstract: An important aspect of examining printed documents for potential forgeries and copyright infringement is the identification of the source printer as it can be helpful for detecting forged documents and ascertaining the leak. This paper proposes a system for classification of source printer from scanned images of printed documents using all the printed letters simultaneously. The proposed system uses local texture patterns-based features and a single classifier for classifying all the printed letters. Letters are extracted from scanned images using connected component analysis followed by morphological filtering without the need of using an optical character recognition. Each letter is sub-divided into a flat region and an edge region, and local tetra patterns are estimated separately for these two regions. A strategically constructed pooling technique is used to extract the final feature vectors. The proposed method has been tested on both a publicly available data set of ten printers, and a new data set of 18 printers scanned at a resolution of 600 as well as 300 dpi printed in four different fonts. The results indicate that the proposed system is capable of simultaneously dealing with all the printed letters and using a single classifier outperforms existing handcrafted feature-based methods. To achieve accuracies similar to that of state-of-art methods, it needs a much smaller number of training pages by using all the printed letters. Citation Count: 30 - Paper ID: 0b6f810f287561ff694a9406c7b319fd8549ca68 Title: Face Recognition Based on Texture Features using Local Ternary Patterns Abstract: Face recognition is one of the important and popular visual recognition problem due to its challenging nature and its diverse set of applications. That‘s why face recognition is attracted by many researchers. Methods based on Local Binary Pattern (LBP) are widely used for face recognition in the literature, and it is sensitive to noise. To address this present paper utilized the powerful local texture descriptor that is less sensitive to noise and more discriminant in uniform regions called as Local Ternary Pattern (LTP). The Uniform Local Binary Pattern (ULBP) derived on LBP treats a large set of LBP under one label called as miscellaneous. This may result some loss of information on LBP and LTP based methods. To address this two Prominent LBP (PLBP) are derived, namely PLBP-Low (L) and PLBP-High (H) on LTP. Based on this the present paper derived eight texture features on facial images. A distance function is used on proposed texture features for effective face recognition. To eliminate most of the effects of illumination changes that are present in human face an efficient preprocessing method is used that preserves the significant appearance details that are needed for face recognition. The present method is experimented on Yale, Indian and American Telephone and Telegraph Company (AT&T) Olivetti Research Laboratory (ORL) data bases and it has given state-of-the-art performance on the three popular datasets. Citation Count: 31 - Paper ID: ecb136a5480807b806315161b3e3dc4d9ce659fb Title: Graph cuts by using local texture features of wavelet coefficient for image segmentation Abstract: This paper proposes an approach to image segmentation using iterated graph cuts based on local texture features of wavelet coefficient. Using multiresolution analysis based on Haar wavelet, low-frequency range (smoothed image) is used for n-link and high-frequency range (local texture features) is used for t-link along with color histogram. The proposed method can segment the object region with noisy edges and colors similar to the background, but heavy texture change. Experimental results illustrate the validity of our method. Citation Count: 7 - Paper ID: 7a1e8441af37588bf014a26077d83d35792fee66 Title: Classification of Fruit in a Box (FIB) Using Hybridization of Color and Texture Features Abstract: The purpose of this paper is to develop an effective classification of fruit in a box by considering the color and texture features from the images. Twenty fruit types with various appearances in color and texture were selected to be analyzed in this study. Although the capability of many color or texture features were previously studied in many researches, each feature cannot be used to identify the fruit type accurately enough for practical use. In this study, we combine six features, i.e., HSV Color Histogram, Color Layout Descriptor (CLD), Color Correlogram, Gray Level Co-occurrence Matrix (GLCM), Local Binary Pattern (LBP), and Neighboring Gray Tone Difference Matrix (NGTDM) to gain high accuracy of fruit-in-a-box classification. An image preprocessing stage is applied to fruit images to prepare the images in good condition. Then, six image features are extracted from each image. Finally, the fruit classification process is adopted through the well-known classification methods such as Decision Tree, Random Forest, k-Nearest Neighbor (kNN), Support Vector Machine (SVM), Logistic Regression classifier, Linear Discriminant Analysis (LDA) classifier, Naïve Bayes classifier, and Multi-Layer Perceptron (MLP). After experiments were tested and evaluated, it shows that, with the appropriate classification method, the hybridization of features yields high accuracy with independence of classification method and effectiveness in the classification of fruit in a box. Citation Count: 4 - Paper ID: 49c17cfa973e32f2d10e8786c93c5979b216fb7e Title: Glioma grade classification using wavelet transform-local binary pattern based statistical texture features and geometric measures extracted from MRI Abstract: ABSTRACT The objective of this study is to perform brain glioma grade classification by identifying an optimal quantitative feature set from conventional magnetic resonance images. In this work, a hybrid feature set comprising of statistical texture and geometric features is computed over entire segmented tumor volume. Discrete wavelet transform (DWT) and local binary pattern (LBP) techniques are combined to extract texture information from segmented tumour volume at multiple resolutions. Statistical texture features comprising of skewness, kurtosis and entropy are then computed from DWT-LBP transformed images. Geometric features are calculated from (i) fractal dimension (FD) of three dimensional (3D) volumes of tumour region, tumour border and tumour skeleton, and (ii) convexity parameters over complete segmented tumour volume. Statistical analysis revealed that extracted texture features are significantly different between high grade (HG) and low grade (LG) glioma patients (p < 0.05). FD-based geometric parameters are significantly higher for HG glioma patients in comparison to LG glioma patients. Our results reflect that HG glioma has more structural complexity than LG glioma. The optimised feature set comprising of DWT-LBP-based texture features and FD-based measures extracted from segmented tumour volume achieved 96% accuracy, 97% sensitivity and 95% specificity for glioma classification with Naive Bayes classifier. Citation Count: 11 - Paper ID: cbaa3f3da459d99297377bd67131277a2ffe70ec Title: Metastasis detection from whole slide images using local features and random forests Abstract: Digital pathology has led to a demand for automated detection of regions of interest, such as cancerous tissue, from scanned whole slide images. With accurate methods using image analysis and machine learning, significant speed‐up, and savings in costs through increased throughput in histological assessment could be achieved. This article describes a machine learning approach for detection of cancerous tissue from scanned whole slide images. Our method is based on feature engineering and supervised learning with a random forest model. The features extracted from the whole slide images include several local descriptors related to image texture, spatial structure, and distribution of nuclei. The method was evaluated in breast cancer metastasis detection from lymph node samples. Our results show that the method detects metastatic areas with high accuracy (AUC = 0.97–0.98 for tumor detection within whole image area, AUC = 0.84–0.91 for tumor vs. normal tissue detection) and that the method generalizes well for images from more than one laboratory. Further, the method outputs an interpretable classification model, enabling the linking of individual features to differences between tissue types. © 2017 International Society for Advancement of Cytometry Citation Count: 46 - Paper ID: a6e4f924cf9a12625e85c974f0ed136b43c2f3b5 Title: A new facial age estimation method using centrally overlapped block based local texture features Abstract: None Citation Count: 7

- Paper ID: 29dc9cf7005047a21d72bfd26ab35bd05c97e2e4 Title: Assessing Texture Features to Classify Coastal Wetland Vegetation from High S... - Paper ID: 29dc9cf7005047a21d72bfd26ab35bd05c97e2e4 Title: Assessing Texture Features to Classify Coastal Wetland Vegetation from High Spatial Resolution Imagery Using Completed Local Binary Patterns (CLBP) Abstract: Coastal wetland vegetation is a vital component that plays an important role in environmental protection and the maintenance of the ecological balance. As such, the efficient classification of coastal wetland vegetation types is key to the preservation of wetlands. Based on its detailed spatial information, high spatial resolution imagery constitutes an important tool for extracting suitable texture features for improving the accuracy of classification. In this paper, a texture feature, Completed Local Binary Patterns (CLBP), which is highly suitable for face recognition, is presented and applied to vegetation classification using high spatial resolution Pleiades satellite imagery in the central zone of Yancheng National Natural Reservation (YNNR) in Jiangsu, China.... - Paper ID: 29dc9cf7005047a21d72bfd26ab35bd05c97e2e4 Title: Assessing Texture Features to Classify Coastal Wetland Vegetation from High Spatial Resolution Imagery Using Completed Local Binary Patterns (CLBP) Abstract: Coastal wetland vegetation is a vital component that plays an important role in environmental protection and the maintenance of the ecological balance. As such, the efficient classification of coastal wetland vegetation types is key to the preservation of wetlands. Based on its detailed spatial information, high spatial resolution imagery constitutes an important tool for extracting suitable texture features for improving the accuracy of classification. In this paper, a texture feature, Completed Local Binary Patterns (CLBP), which is highly suitable for face recognition, is presented and applied to vegetation classification using high spatial resolution Pleiades satellite imagery in the central zone of Yancheng National Natural Reservation (YNNR) in Jiangsu, China. To demonstrate the potential of CLBP texture features, Grey Level Co-occurrence Matrix (GLCM) texture features were used to compare the classification. Using spectral data alone and spectral data combined with texture features, the image was classified using a Support Vector Machine (SVM) based on vegetation types. The results show that CLBP and GLCM texture features yielded an accuracy 6.50% higher than that gained when using only spectral information for vegetation classification. However, CLBP showed greater improvement in terms of classification accuracy than GLCM for Spartina alterniflora. Furthermore, for the CLBP features, CLBP_magnitude (CLBP_m) was more effective than CLBP_sign (CLBP_s), CLBP_center (CLBP_c), and CLBP_s/m or CLBP_s/m/c. These findings suggest that the CLBP approach offers potential for vegetation classification in high spatial resolution images. Citation Count: 35 - Paper ID: 810ae452a3a1f673ea241bd540f9551b2996ed5b Title: Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet Abstract: Deep Neural Networks (DNNs) excel on many complex perceptual tasks but it has proven notoriously difficult to understand how they reach their decisions. We here introduce a high-performance DNN architecture on ImageNet whose decisions are considerably easier to explain. Our model, a simple variant of the ResNet-50 architecture called BagNet, classifies an image based on the occurrences of small local image features without taking into account their spatial ordering. This strategy is closely related to the bag-of-feature (BoF) models popular before the onset of deep learning and reaches a surprisingly high accuracy on ImageNet (87.6% top-5 for 33 x 33 px features and Alexnet performance for 17 x 17 px features). The constraint on local features makes it straight-forward to analyse how exactly each part of the image influences the classification. Furthermore, the BagNets behave similar to state-of-the art deep neural networks such as VGG-16, ResNet-152 or DenseNet-169 in terms of feature sensitivity, error distribution and interactions between image parts. This suggests that the improvements of DNNs over previous bag-of-feature classifiers in the last few years is mostly achieved by better fine-tuning rather than by qualitatively different decision strategies. Citation Count: 507 - Paper ID: 37b0abdea4a976062c069d0652dac38af943ba68 Title: Block-based cloud classification with statistical features and distribution of local texture features Abstract: Abstract. This work performs cloud classification on all-sky images. To deal with mixed cloud types in one image, we propose performing block division and block-based classification. In addition to classical statistical texture features, the proposed method incorporates local binary pattern, which extracts local texture features in the feature vector. The combined feature can effectively preserve global information as well as more discriminating local texture features of different cloud types. The experimental results have shown that applying the combined feature results in higher classification accuracy compared to using classical statistical texture features. In our experiments, it is also validated that using block-based classification outperforms classification on the entire images. Moreover, we report the classification accuracy using different classifiers including the k-nearest neighbor classifier, Bayesian classifier, and support vector machine. Citation Count: 30 - Paper ID: 10e58a4fd2a23840d50ff4c540ad6ad21a602f9e Title: Content-based image retrieval using local texture features in distributed environment Abstract: In this paper, we propose novel content-based image retrieval (CBIR) algorithms using Local Octa Patterns (LOtP), Local Hexadeca Patterns (LHdP) and Direction Encoded Local Binary Pattern (DELBP). LOtP and LHdP encode the relationship between center pixel and its neighbors based on the pixels’ direction obtained by considering the horizontal, vertical and diagonal pixels for derivative calculations. In DELBP, direction of a referenced pixel is determined by considering every neighboring pixel for derivative calculations which results in 256 directions. For this resultant direction encoded image, we have obtained LBP which is considered as feature vector. The proposed method’s performance is compared to that of Local Tetra Patterns (LTrP) using benchmark image databases viz., Corel 1000 (DB1) and Brodatz textures (DB2). Performance analysis shows that LOtP improves the average precision from 59.31% to 64.36% on DB1, and from 83.24% to 85.95% on DB2, LHdP improves it to 65.82% on DB1 and to 87.49% on DB2 and DELBP improves it to 60.35% on DB1 and to 86.12% on DB2 as compared to that of LTrP. Also, DELBP reduces the feature vector length by 66.62% as compared to that of LTrP. To reduce the retrieval time, the proposed algorithms are implemented on a Hadoop cluster consisting of 116 nodes and tested using Corel 10K (DB3), Mirflickr 100,000 (DB4) and ImageNet 511,380 (DB5) databases. Citation Count: 10 - Paper ID: e6d9ee691c4aa1eb991bf36cda055a7983d57cf1 Title: Local epigenomic state cannot discriminate interacting and non-interacting enhancer–promoter pairs with high accuracy Abstract: We report an overfitting issue in recent machine learning formulations of the enhancer-promoter interaction problem arising from the fact that many enhancer-promoter pairs share features. Cross- fold validation schemes which do not correctly separate these feature sharing enhancer-promoter pairs into one test set report high accuracy, which is actually due to overfitting. Cross-fold validation schemes which properly segregate pairs with shared features show markedly reduced ability to predict enhancer-promoter interactions from epigenomic state. Parameter scans with multiple models indicate that local epigenomic features of individual pairs of enhancers and promoters cannot distinguish those pairs that interact from those which do with high accuracy, suggesting that additional information is required to predict enhancer-promoter interactions. Citation Count: 38 - Paper ID: 10d7734ccfd6061c13d2f50f7eea5a6162479c36 Title: Single Classifier-Based Passive System for Source Printer Classification Using Local Texture Features Abstract: An important aspect of examining printed documents for potential forgeries and copyright infringement is the identification of the source printer as it can be helpful for detecting forged documents and ascertaining the leak. This paper proposes a system for classification of source printer from scanned images of printed documents using all the printed letters simultaneously. The proposed system uses local texture patterns-based features and a single classifier for classifying all the printed letters. Letters are extracted from scanned images using connected component analysis followed by morphological filtering without the need of using an optical character recognition. Each letter is sub-divided into a flat region and an edge region, and local tetra patterns are estimated separately for these two regions. A strategically constructed pooling technique is used to extract the final feature vectors. The proposed method has been tested on both a publicly available data set of ten printers, and a new data set of 18 printers scanned at a resolution of 600 as well as 300 dpi printed in four different fonts. The results indicate that the proposed system is capable of simultaneously dealing with all the printed letters and using a single classifier outperforms existing handcrafted feature-based methods. To achieve accuracies similar to that of state-of-art methods, it needs a much smaller number of training pages by using all the printed letters. Citation Count: 30 - Paper ID: ac467c382b51574c70033a2786f633fc4a83cfaf Title: An Automated Age-Related Macular Degeneration Classification Based on Local Texture Features in Optical Coherence Tomography Angiography Abstract: None Citation Count: 3 - Paper ID: 49c17cfa973e32f2d10e8786c93c5979b216fb7e Title: Glioma grade classification using wavelet transform-local binary pattern based statistical texture features and geometric measures extracted from MRI Abstract: ABSTRACT The objective of this study is to perform brain glioma grade classification by identifying an optimal quantitative feature set from conventional magnetic resonance images. In this work, a hybrid feature set comprising of statistical texture and geometric features is computed over entire segmented tumor volume. Discrete wavelet transform (DWT) and local binary pattern (LBP) techniques are combined to extract texture information from segmented tumour volume at multiple resolutions. Statistical texture features comprising of skewness, kurtosis and entropy are then computed from DWT-LBP transformed images. Geometric features are calculated from (i) fractal dimension (FD) of three dimensional (3D) volumes of tumour region, tumour border and tumour skeleton, and (ii) convexity parameters over complete segmented tumour volume. Statistical analysis revealed that extracted texture features are significantly different between high grade (HG) and low grade (LG) glioma patients (p < 0.05). FD-based geometric parameters are significantly higher for HG glioma patients in comparison to LG glioma patients. Our results reflect that HG glioma has more structural complexity than LG glioma. The optimised feature set comprising of DWT-LBP-based texture features and FD-based measures extracted from segmented tumour volume achieved 96% accuracy, 97% sensitivity and 95% specificity for glioma classification with Naive Bayes classifier. Citation Count: 11 - Paper ID: a6e4f924cf9a12625e85c974f0ed136b43c2f3b5 Title: A new facial age estimation method using centrally overlapped block based local texture features Abstract: None Citation Count: 7 - Paper ID: 049d6ad402ccad747fd759847b7005fdcbcdee1f Title: Local texture descriptors and projection histogram based handwritten Meitei Mayek character recognition Abstract: None Citation Count: 6

Published as a conference paper at ICLR 2019 APPROXIMATING CNN S WITH BAG-OF-LOCAL - FEATURES MODELS WORKS SURPRISINGLY WELL ONIMAGE NET Wie... Published as a conference paper at ICLR 2019 APPROXIMATING CNN S WITH BAG-OF-LOCAL - FEATURES MODELS WORKS SURPRISINGLY WELL ONIMAGE NET Wieland Brendel and Matthias Bethge Eberhard Karls University of Tübingen, Germany Werner Reichardt Centre for Integrative Neuroscience, Tübingen, Germany Bernstein Center for Computational Neuroscience, Tübingen, Germany {wieland.brendel, matthias.bethge}@bethgelab.org ABSTRACT Deep Neural Networks (DNNs) excel on many complex perceptual tasks but it has proven notoriously difficult to understand how they reach their decisions. We here introduce a high-performance DNN architecture on ImageNet whose decisions are considerably easier to explain. Our model, a simple variant of the ResNet- 50 architecture called BagNet, classifies an image based on the occurrences of small local image features without taking into account their spatial ordering. This strategy is closely related to the bag-of-feature (BoF) models popular before the onset of deep learning and... Published as a conference paper at ICLR 2019 APPROXIMATING CNN S WITH BAG-OF-LOCAL - FEATURES MODELS WORKS SURPRISINGLY WELL ONIMAGE NET Wieland Brendel and Matthias Bethge Eberhard Karls University of Tübingen, Germany Werner Reichardt Centre for Integrative Neuroscience, Tübingen, Germany Bernstein Center for Computational Neuroscience, Tübingen, Germany {wieland.brendel, matthias.bethge}@bethgelab.org ABSTRACT Deep Neural Networks (DNNs) excel on many complex perceptual tasks but it has proven notoriously difficult to understand how they reach their decisions. We here introduce a high-performance DNN architecture on ImageNet whose decisions are considerably easier to explain. Our model, a simple variant of the ResNet- 50 architecture called BagNet, classifies an image based on the occurrences of small local image features without taking into account their spatial ordering. This strategy is closely related to the bag-of-feature (BoF) models popular before the onset of deep learning and reaches a surprisingly high accuracy on ImageNet (87.6% top-5 for 33 33 px features and Alexnet performance for 17 17 px features). The constraint on local features makes it straight-forward to analyse how exactly each part of the image influences the classification. Furthermore, the BagNets behave similar to state-of-the art deep neural networks such as VGG-16, ResNet-152 or DenseNet-169 in terms of feature sensitivity, error distribution and interactions between image parts. This suggests that the improvements of DNNs over previous bag-of-feature classifiers in the last few years is mostly achieved by better fine-tuning rather than by qualitatively different decision strategies. 1 I NTRODUCTION A big obstacle in understanding the decision-making of DNNs is due to the complex dependencies between input and hidden activations: for one, the effect of any part of the input on a hidden activation depends on the state of many other parts of the input. Likewise, the role of a hidden unit on downstream representations depends on the activity of many other units. This dependency makes it extremely difficult to understand how DNNs reach their decisions. To circumvent this problem we here formulate a new DNN architecture that is easier to interpret by design . Our architecture is inspired by bag-of-feature (BoF) models which — alongside extensions such as VLAD encoding or Fisher Vectors — have been the most successful approaches to large-scale object recognition before the advent of deep learning (up to 75% top-5 on ImageNet) and classify images based on the counts, but not the spatial relationships, of a set of local image features. This structure makes the decisions of BoF models particularly easy to explain. To be concise, throughout this manuscript the concept of interpretability refers to the way in which evidence from small image patches is integrated to reach an image-level decision. While basic BoF models perform just a simple and transparent spatial aggregate of the patch-wise evidences, DNNs non-linearly integrate information across the whole image. In this paper we show that it is possible to combine the performance and flexibility of DNNs with the interpretability of BoF models, and that the resulting model family (called BagNets ) is able to reach high accuracy on ImageNet even if limited to fairly small image patches. Given the simplicity of BoF models we imagine many use cases for which it can be desirable to trade a bit of accuracy for better interpretability, just as this is common e.g. for linear function approximation. This includes diagnosing failure cases (e.g. adversarial examples) or non-iid. settings (e.g. domain transfer), 1arXiv:1904.00760v1 [cs.CV] 20 Mar 2019Published as a conference paper at ICLR 2019 scuba diver 24 89scuba diverwreck 21 4wreckcoral reef 21 4 coral reefloggerhead 20 1 loggerheadp =softmaxBagNets extract class activations (logits) on each patchEvaluating all patches yields one heatmap per classAccumulating activations over space yields total class evidenceValidation performance of BagNetsPearson correlation between BagNets and VGG=16 class activations (logits)A B C Figure 1: Deep bag-of-features models (BagNets). (A) The models extract features from small image patches which are each fed into a linear classifier yielding one logit heatmap per class. These heatmaps are averaged across space and passed through a softmax to get the final class probabilities. (B) Top-5 ImageNet performance over patch size. (C) Correlation with logits of VGG-16. benchmarking diagnostic tools (e.g. attribution methods) or serving as interpretable parts of a computer vision pipeline (e.g. with a relational network on top of the local features). In addition, we demonstrate similarities between the decision-making behaviour of BagNets and popular DNNs in computer vision. These similarities suggest that current network architectures base their decisions on a large number of relatively weak and local statistical regularities and are not sufficiently encouraged - either through their architecture, training procedure or task specification - to learn more holistic features that can better appreciate causal relationships between different parts of the image. 2 N ETWORK ARCHITECTURE We here recount the main elements of a classic bag-of-features model before introducing the simpler DNN-based BagNets in the next paragraph. Bag-of-feature representations can be described by analogy to bag-of-words representations. With bag-of-words, one counts the number of occurrences of words from a vocabulary in a document. This vocabulary contains important words (but not common ones like "and" or "the") and word clusters (i.e. semantically similar words like "gigantic" and "enormous" are subsumed). The counts of each word in the vocabulary are assembled as one long term vector . This is called the bag-of-words document representation because all ordering of the words is lost. Likewise, bag-of-feature representations are based on a vocabulary of visual words which represent clusters of local image features. The term vector for an image is then simply the number of occurrences of each visual word in the vocabulary. This term vector is used as an input to a classifier (e.g. SVM or MLP). Many successful image classification models have been based on this pipeline (Csurka et al., 2004; Jurie & Triggs, 2005; Zhang et al., 2007; Lazebnik et al., 2006), see O’Hara & Draper (2011) for an up-to-date overview. BoF models are easy to interpret if the classifier on top of the term vector is linear. In this case the influence of a given part of the input on the classifier is independent of the rest of the input. Based on this insight we construct a linear DNN-based BoF model as follows (see Figure 1): first, we infer a 2048 dimensional feature representation from each image patch of size q q pixels using multiple stacked ResNet blocks and apply a linear classifier to infer the class evidence for each patch (heatmaps ). We average the class evidence across all patches to infer the image-level class evidence (logits). This structure differs from other ResNets (He et al., 2015) only in the replacement of many 33 by 11 convolutions, thereby limiting the receptive field size of the topmost convolutional layer to qq pixels (see Appendix for details). There is no explicit assignment to visual words. This could be added through a sparse projection into a high-dimensional embedding but we did not see benefits for interpretability. We denote the resulting architecture as BagNet- qand testq2[9;17;33]. 2Published as a conference paper at ICLR 2019 Original BagNet-9 BagNet-17 BagNet-33 Figure 2: Heatmaps showing the class evidence extracted from of each part of the image. The spatial sum over the evidence is the total class evidence. Note that an important ingredient of our model is the linear classifier on top of the local feature representation. The word linear here refers to the combination of a linear spatial aggregation (a simple average) and a linear classifier on top of the aggregated features. The fact that the classifier and the spatial aggregation are both linear and thus interchangeable allows us to pinpoint exactly how evidence from local image patches is integrated into one image-level decision. 3 R ELATED LITERATURE BoF models and DNNs There are some model architectures that fuse elements from DNNs and BoF models. Predominantly, DNNs were used to replace the previously hand-tuned feature extraction stage in BoF models, often using intermediate or higher layer features of pretrained DNNs (Feng et al., 2017; Gong et al., 2014; Ng et al., 2015; Mohedano et al., 2016; Cao et al., 2017; Khan et al., 2016) for tasks such as image retrieval or geographical scene classification. Other work has explored how well insights from DNN training (e.g. data augmentation) transfer to the training of BoF and Improved Fisher Vector models (Chatfield et al., 2014) and how SIFT and CNN feature descriptions perform (Babenko & Lempitsky, 2015). In contrast, our proposed BoF model architecture is simpler and closer to standard DNNs used for object recognition while still maintaining the interpretability of linear BoF models with local features. Furthermore, to our knowledge this is the first work that explores the relationship between the decision-making of BoF and DNN models. Interpretable DNNs Our work is most closely related to approaches that use DNNs in conjunction with more interpretable elements. Pinheiro & Collobert (2014) adds explicit labelling of single pixels before the aggregation to an image-level label. The label of each pixel, however, is still inferred from the whole image, making the pixel assignments difficult to interpret. Xiao et al. (2015) proposed a multi-step approach combining object-, part- and domain-detectors to reach a classification decision. In this process object-relevant patches of variable sizes are extracted. In contrast, our approach is much simpler, reaches higher accuracy and is easier to interpret. Besides pixel-level attention-based mechanisms there are several attempts to make the evidence accumulation more interpretable. Hinton et al. (2015) introduced soft decision trees that are trained on the predictions of neural networks. While this increases performance of decision trees, the gap to neural networks on data sets like ImageNet is still large. In Li et al. (2017) an autoencoder architecture is combined with a shallow classifier based on prototype representations. Chen et al. (2018) uses a similar approach but is based on a convolutional architecture to extract class-specific prototype patches. The interpretation of the prototype-based classification, however, is difficult because only the L2 norm between the prototypes and the extracted latent representations is considered1. Finally, the class activation maps by Zhou et al. (2015) share similarities to our approach as they also use a CNN with global average pooling and a linear classifier in order to extract class-specific heatmaps. However, their latent representations are 1Just because two images have a similar latent representation does not mean that they share any similarities in terms of human-interpretable features. 3Published as a conference paper at ICLR 2019 BagNet-33 BagNet-17 BagNet-9tench tench tenchhand-held computer hand-held computer hand-held computerbook jacket groom snailmountain bike football helmet flamingo Figure 3: Most informative image patches for BagNets. For each class (row) and each model (column) we plot two subrows: in the top subrow we show patches that caused the highest logit outputs for the given class across all validation images with that label. Patches in the bottom subrow are selected in the same way but from all validation images with a different label (highlighting errors). extracted from the whole image and it is unclear how the heatmaps in the latent space are related to the pixel space. In our approach the CNN representations are restricted to very small image patches, making it possible to trace exactly how each image patch contributes the final decision. Scattering networks Another related work by Oyallon et al. (2017) uses a scattering network with small receptive fields (14 x 14 pixels) in conjunction with a two-layer Multilayer Perceptron or a ResNet-10 on top of the scattering network. This approach reduces the overall depth of the model compared to ResNets (with matched classification accuracy) but does not increase interpretability (because of the non-linear classifier on top of the local scattering features). A set of superficially similar but unrelated approaches are region proposal models (Wei et al., 2016; Tang et al., 2017; 2016; Arandjelovic et al., 2015). Such models typically use the whole image to infer smaller image regions with relevant objects. These regions are then used to extract a spatially aligned subset of features from the highest DNN layer (so information is still integrated far beyond the proposed image patch). Our approach does not rely on region proposals and extracts features only from small local regions. 4 R ESULTS In the first two subsections we investigate the classification performance of BagNets for different patch sizes and demonstrate insights we can derive from its interpretable structure. Thereafter we compare the behaviour of BagNets with several widely used high-performance DNNs (e.g. VGG-16, ResNet-50, DenseNet-169) and show evidence that their decision-making shares many similarities. 4.1 A CCURACY &RUNTIME OF BAGNETS ON IMAGE NET We train the BagNets directly on ImageNet (see Appendix for details). Surprisingly, patch sizes as small as 1717 pixels suffice to reach AlexNet (Krizhevsky et al., 2012) performance (80.5% top-5 performance) while patches sizes 33 33 pixels suffice to reach close to 87.6%. We also compare the runtime of BagNet-q ( q= 33;17;9) in inference mode with images of size 3224224and batch size 64 against a vanilla ResNet-50. Across all receptive field sizes BagNets reach around 155 images/s for BagNets compared to 570 images/s for ResNet-50. The difference in runtime can be attributed to the reduced amount of downsampling in BagNets compared to ResNet-50. 4.2 E XPLAINING DECISIONS For each qq patch the model infers evidence for each ImageNet classes, thus yielding a high- resolution and very precise heatmap that shows which parts of the image contributes most to certain decisions. We display these heatmaps for the predicted class for ten randomly chosen test images in 4Published as a conference paper at ICLR 2019 Figure 2. Clearly, most evidence lies around the shapes of objects (e.g. the crip or the paddles) or certain predictive image features like the glowing borders of the pumpkin. Also, for animals eyes or legs are important. It’s also notable that background features (like the forest in the deer image) are pretty much ignored by the BagNets. original true class predicted class Figure 4: Images misclassified by BagNet- 33 and VGG-16 with heatmaps for true and predicted label and the most predictive im- age patches. Class probability reported for BagNet-33 (left) and VGG (right).Next we pick a class and run the BagNets across all validation images to find patches with the most class evidence. Some of these patches are taken from im- ages of that class (i.e. they carry "correct" evidence) while other patches are from images of another class (i.e. these patches can lead to misclassifications). In Figure 3 we show the top-7 patches from both cor- rect and incorrect images for several classes (rows) and different BagNets (columns). This visualisation yields many insights: for example, book jackets are identified mainly by the text on the cover, leading to confusion with other text on t-shirts or websites. Similarly, keys of a typewriter are often interpreted as evidence for handheld computers. The tench class, a large fish species, is often identified by fingers on front of a greenish background. Closer inspection re- vealed that tench images typically feature the fish hold up like a trophy, thus making the hand and fingers holding it a very predictive image feature. Flamingos are detected by their beaks, which makes them easy to confuse with other birds like storks, while grooms are primarily identified by the transition from suit to neck, an image feature present in many other classes. In Figure 4 we analyse images misclassified by both BagNet-33 and VGG-16. In the first example the ground-truth class "cleaver" was confused with "granny smith" because of the green cucumber at the top of the image. Looking at the three most predictive patches plotted alongside each heatmap, which show the apple-like edges of the green cucumber pieces, this choice looks comprehensible. Similarly, the local patches in the "thimble" image resemble a gas mask if viewed in isolation. The letters in the "miniskirt" image are very salient, thus leading to the "book jacket" prediction while in the last image the green blanket features a glucamole-like texture. 4.3 C OMPARING THE DECISION -MAKING OF BAGNETS AND HIGH -PERFORMANCE DNN S In the next paragraphs we investigate how similar the decision-making of BagNets is to high- performance DNNs like VGG-16, ResNet-50, ResNet-152 and DenseNet-169. There is no single answer or number, partially because we lack a sensible distance metric between networks. One can original texturised images original texturised images Figure 5: Examples of original and texturised images. A vanilla VGG-16 still reaches high accuracy on the texturised images while humans suffer greatly from the loss of global shapes in many images. 5Published as a conference paper at ICLR 2019 A B C Figure 6: Interaction of spatially separated image parts. (A) Changes in class-evidence when single image patches are masked (centre) versus change when all patches are masked simultaneously (right). For linear BoF models both terms are the same. (B) Masking regions for different patch sizes. (C) Correlation between both terms for different DNNs over different patch sizes. Interactions are greatly depressed for image features larger than 30 30 px. compare the pearson correlation between logits (for VGG-16, BagNet-9/17/33 reach 0.70 / 0.79 / 0.88 respectively, see Figure 1C), but this number can only give a first hint as it does not investigate the specific process that led to the decision. However, the decision-making of BagNets does feature certain key characteristics that we can compare to other models. Image Scrambling One core component of the bag-of-feature networks is the neglect of the spatial relationships between image parts. In other words, scrambling the parts across the image while keeping their counts constant does not change the model decision. Is the same true for current computer vision models like VGG or ResNets? Unfortunately, due to the overlapping receptive fields it is generally not straight-forward to scramble an image in a way that leaves the feature histograms invariant. For VGG-16 an algorithm that comes close to this objective is the popular texture synthesis algorithm based on the Gram features of the hidden layer activations (Gatys et al., 2015), Figure 5. For humans, the scrambling severely increases the difficulty of the task while the performance of VGG-16 is little affected (90.1% on clean versus 79.4% on texturised image). This suggests that VGG, in stark contrast to humans, does not rely on global shape integration for perceptual discrimination but rather on statistical regularities in the histogram of local image features. It is well known by practioners that the aforementioned texture synthesis algorithm does not work for ResNet- and DenseNet architectures, the reasons of which are not yet fully understood. Spatially distinct image manipulations do not interact For BoF models with a linear (but not non-linear!) classifier we do not only expect invariance to the spatial arrangement of image parts, but also that the marginal presence or absence of an image part always has the same effect on the 6Published as a conference paper at ICLR 2019 evidence accumulation (i.e. is independent of the rest of the image). In other words, for a BoF model an image with five unconnected wheels (and nothing else) would carry more evidence for class "bike" than a regular photo of a bicycle; a linear BoF model simply ignores whether there is also a frame and a saddle. More precisely, let `model(x)be the class evidence (logit) as a function of the input xand let ibe spatially separated and non-overlapping input modifications. For a BagNet-q it holds that `model(x)`model(x+X ii) =X i(`model(x)`model(x+i)); (1) as long as the modifications are separated by more than qpixels. We use the Pearson correlation between the LHS and RHS of eq. (1) as a measure of non-linear interactions between image parts. In our experiments we partition the image into a grid of non-overlapping square-sized patches with patch sizeq. We then replace every second patch in every second row (see Figure 6B) with its DC component (the spatial channel average) both in isolation (RHS of eq. (1)) and in combination (LHS of eq. (1)), see Figure 6A. This ensures that the masked patches are spaced by qpixels and that always around 1/4 of the image is masked. Since most objects fill much of the image, we can expect that the masking will remove many class-predictive image features. We measure the Pearson correlation between the LHS and RHS of eq. 1 for different patch sizes qand DNN models (Figure 6C). The results (Figure 6C) show that VGG-16 exhibits few interactions between image parts spaced by more than 30 pixels. The interactions increase for deeper and more performant architectures. Error distribution In Figure 7 we plot the top-5 accuracy within each ImageNet class of BagNet- 33 against the accuracy of regular DNNs. For comparison we also plot VGG-11 against VGG-16. The analysis reveals that the error distribution is fairly consistent between models. Spatial sensitivity To see whether BagNets and DNNs use similar image parts for image classi- fication we follow Zintgraf et al. (2017) and test how the prediction of DNNs is changing when we mask the most predictive image parts. In Figure 8 (top) we compare the decrease in predicted class probability for an increasing number of masked 8 8 patches. The masking locations are determined by the heatmaps of BagNets which we compare against random maskings as well as several popular attribution techniques (Baehrens et al., 2010; Sundararajan et al., 2017; Kindermans et al., 2018; Shrikumar et al., 2017) (we use the implementations of DeepExplain (Ancona et al., 2017)) which compute heatmaps directly in the tested models. Notice that these attribution methods have an advantage because they compute heatmaps knowing everything about the models (white-box setting). Nonetheless, the heatmaps from BagNets turn out to be more predictive for class-relevant image parts (see also Table 1). In other words, image parts that are relevant to BagNets are similarly relevant for the classification of normal DNNs. VGG-16 is most affected by the masking of local patches while deeper and more performant architectures are more robust to the relatively small masks, which again suggests that deeper architectures take into account larger spatial relationships. 5 D ISCUSSION & O UTLOOK In this paper we introduced and analysed a novel interpretable DNN architecture — coined BagNets — that classifies images based on linear bag-of-local-features representations. The results demonstrate that even complex perceptual tasks like ImageNet can be solved just based on small image features and without any notion of spatial relationships. In addition we showed that the key properties of BagNets, in particlar invariance to spatial relationships as well as weak interactions between image features, are also present to varying degrees in many common computer vision models like ResNet-50 40 65 100 VGG-16 [%, top-5]4065100VGG-11 [%, top-5] 40 65 100 VGG-16 [%, top-5]4065100BagNet-33 [%, top-5] 40 65 100 ResNet-50 [%, top-5] 40 65 100 ResNet-152 [%, top-5] 40 65 100 DenseNet-169 [%, top-5] Figure 7: Scatter plots of class-conditional top-5 errors for different models. 7Published as a conference paper at ICLR 2019 Original Saliency maps Int. Gradients Epsilon-LRP DeepLIFT (rescale) BagNet-9 BagNet-17 BagNet-33 Figure 8: Similarity of image features used for object classification. (Top) Decrease of leading class probability in VGG-16, ResNet-50, ResNet-152 and DenseNet-169 if increasingly more patches are masked according to the heatmaps of BagNets and several popular attribution methods. The faster the decrease the more closely does the heatmap highlight image parts relevant for the model decisions making. Image parts relevant to the BagNets turn out to be similarly relevant for all models and outperform post-hoc attribution methods. (Bottom) The first four heatmaps show attributions computed on VGG-16, the other three heatmaps show the class evidence of BagNets. Sali- encyInt. Grad.-LRPDeep LIFTBN-9 BN-17 BN-33 white-box black-box VGG-16 0.369 0.250 0.326 0.162 0.158 0.151 0.193 ResNet-50 0.528 0.492 0.545 0.379 0.281 0.263 0.291 ResNet-152 0.602 0.580 0.614 0.479 0.394 0.371 0.393 DenseNet-169 0.589 0.515 0.571 0.423 0.339 0.326 0.359 Table 1: Average probability of leading class after masking the 100 patches (8 8 pixels) with the highest attribution according to different heatmaps (columns). or VGG-16, suggesting that the decision-making of many DNNs trained on ImageNet follows at least in part a similar bag-of-feature strategy. In contrast to the perceived “leap” in performance from bag-of-feature models to deep neural networks, the representations learnt by DNNs may in the end still be similar to the pre-deep learning era. VGG-16 is particularly close to bag-of-feature models, as demonstrated by the weak interactions (Figure 6) and the sensitivity to the same small image patches as BagNets (Figure 8). Deeper networks, on the other hand, exhibit stronger nonlinear interactions between image parts and are less sensitive to local maskings. This might explain why texturisation (Figure 5) works well in VGG-16 but fails for ResNet- and DenseNet-architectures. Clearly, ImageNet alone is not sufficient to force DNNs to learn more physical and causal repre- sentation of the world — simply because such a representation is not necessary to solve the task (local image features are enough). This might explain why DNNs generalise poorly to distribution shifts: a DNN trained on natural images has learnt to recognize the textures and local image features 8Published as a conference paper at ICLR 2019 associated with different objects (like the fur and eyes of a cat or the keys of a typewriter) and will inevitably fail if presented with cartoon-like images as they lack the key local image features upon which it bases its decisions. One way forward is to define novel tasks that cannot be solved using local statistical regularities. Here the BagNets can serve as a way to evaluate a lower-bound on the task performance as a function of the observable length-scales. Furthermore, BagNets can be an interesting tool in any application in which it is desirable to trade some accuracy for better interpretability. For example, BagNets can make it much easier to spot the relevant spatial locations and image features that are predictive of certain diseases in medical imaging. Likewise, they can serve as diagnostic tools to benchmark feature attribution techniques since ground-truth attributions are directly available. BagNets can also serve as interpretable parts of a larger computer vision pipeline (e.g. in autonomous cars) as they make it easier to understand edge and failure cases. We released the pretrained BagNets (BagNet-9, BagNet- 17 and BagNet-33) for PyTorch and Keras at https://github.com/wielandbrendel/ bag-of-local-features-models . Taken together, DNNs might be more powerful than previous hand-tuned bag-of-feature algorithms in discovering weak statistical regularities, but that does not necessarily mean that they learn substantially different representations. We hope that this work will encourage and inspire future work to adapt tasks, architectures and training algorithms to encourage models to learn more causal models of the world. ACKNOWLEDGMENTS This work has been funded, in part, by the German Research Foundation (DFG CRC 1233 on “Robust Vision”) as well as by the Intelligence Advanced Research Projects Activity (IARPA) via Department of Interior / Interior Business Center (DoI/IBC) contract number D16PC00003. REFERENCES Marco Ancona, Enea Ceolini, Cengiz Öztireli, and Markus Gross. Towards better understanding of gradient-based attribution methods for deep neural networks, 2017. URL http://arxiv. org/abs/1711.06104 . Relja Arandjelovic, Petr Gronát, Akihiko Torii, Tomás Pajdla, and Josef Sivic. Netvlad: CNN architecture for weakly supervised place recognition. CoRR , abs/1511.07247, 2015. URL http: //arxiv.org/abs/1511.07247 . Artem Babenko and Victor S. Lempitsky. Aggregating deep convolutional features for image retrieval. CoRR , abs/1510.07493, 2015. URL http://arxiv.org/abs/1510.07493 . David Baehrens, Timon Schroeter, Stefan Harmeling, Motoaki Kawanabe, Katja Hansen, and Klaus- Robert Müller. How to explain individual classification decisions. Journal of Machine Learning Research , 11:1803–1831, 2010. Jiewei Cao, Zi Huang, and Heng Tao Shen. Local deep descriptors in bag-of-words for image retrieval. In Proceedings of the on Thematic Workshops of ACM Multimedia 2017 , Thematic Workshops ’17, pp. 52–58, New York, NY , USA, 2017. ACM. ISBN 978-1-4503-5416-5. doi: 10.1145/3126686.3127018. Ken Chatfield, Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Return of the devil in the details: Delving deep into convolutional nets. In Michel François Valstar, Andrew P. French, and Tony P. Pridmore (eds.), BMVC . BMV A Press, 2014. Chaofan Chen, Oscar Li, Alina Barnett, Jonathan Su, and Cynthia Rudin. This looks like that: deep learning for interpretable image recognition. CoRR , abs/1806.10574, 2018. URL http: //arxiv.org/abs/1806.10574 . G. Csurka, C. Bray, C. Dance, and L. Fan. Visual categorization with bags of keypoints. Workshop on Statistical Learning in Computer Vision, ECCV , pp. 1–22, 2004. 9Published as a conference paper at ICLR 2019 Jiangfan Feng, Yuanyuan Liu, and Lin Wu. Bag of visual words model with deep spatial features for geographical scene classification. Comp. Int. and Neurosc. , 2017:5169675:1–5169675:14, 2017. Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge. Texture synthesis and the controlled generation of natural stimuli using convolutional neural networks. CoRR , abs/1505.07376, 2015. Yunchao Gong, Liwei Wang, Ruiqi Guo, and Svetlana Lazebnik. Multi-scale orderless pooling of deep convolutional activation features. In David J. Fleet, Tomás Pajdla, Bernt Schiele, and Tinne Tuytelaars (eds.), ECCV (7) , volume 8695 of Lecture Notes in Computer Science , pp. 392–407. Springer, 2014. ISBN 978-3-319-10583-3. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CoRR , abs/1512.03385, 2015. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network, 2015. Frédéric Jurie and Bill Triggs. Creating efficient codebooks for visual recognition. In ICCV , pp. 604–610, 2005. Fahad Shahbaz Khan, Joost van de Weijer, Rao Muhammad Anwer, Andrew D. Bagdanov, Michael Felsberg, and Jorma Laaksonen. Scale coding bag of deep features for human attribute and action recognition. CoRR , abs/1612.04884, 2016. Pieter-Jan Kindermans, Kristof T Schütt, Maximilian Alber, Klaus-Robert Müller, Dumitru Er- han, Been Kim, and Sven Dähne. Learning how to explain neural networks: Patternnet and patternattribution. In 6th International Conference on Learning Representations , 2018. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolu- tional neural networks. In Advances in neural information processing systems , pp. 1097–1105, 2012. Svetlana Lazebnik, Cordelia Schmid, and Jean Ponce. Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. 2006. Oscar Li, Hao Liu, Chaofan Chen, and Cynthia Rudin. Deep learning for case-based reasoning through prototypes: A neural network that explains its predictions. CoRR , abs/1710.04806, 2017. URLhttp://arxiv.org/abs/1710.04806 . Eva Mohedano, Kevin McGuinness, Noel E. O’Connor, Amaia Salvador, Ferran Marques, and Xavier Giro-i Nieto. Bags of local convolutional features for scalable instance search. In Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval , ICMR ’16, pp. 327–331, New York, NY , USA, 2016. ACM. ISBN 978-1-4503-4359-6. doi: 10.1145/2911996.2912061. URLhttp://doi.acm.org/10.1145/2911996.2912061 . Joe Yue-Hei Ng, Fan Yang, and Larry S. Davis. Exploiting local features from deep networks for image retrieval. In CVPR Workshops , pp. 53–61. IEEE Computer Society, 2015. ISBN 978-1-4673-6759-2. Stephen O’Hara and Bruce A. Draper. Introduction to the bag of features paradigm for image classification and retrieval. abs/1101.3354, 2011. Edouard Oyallon, Eugene Belilovsky, and Sergey Zagoruyko. Scaling the scattering transform: Deep hybrid networks. CoRR , abs/1703.08961, 2017. URL http://arxiv.org/abs/1703. 08961 . Pedro H. O. Pinheiro and Ronan Collobert. Weakly supervised semantic segmentation with convolu- tional networks. CoRR , abs/1411.6228, 2014. URL http://arxiv.org/abs/1411.6228 . Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. CoRR , abs/1704.02685, 2017. Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. CoRR , abs/1703.01365, 2017. 10Published as a conference paper at ICLR 2019 Peng Tang, Xinggang Wang, Baoguang Shi, Xiang Bai, Wenyu Liu, and Zhuowen Tu. Deep fishernet for object classification. CoRR , abs/1608.00182, 2016. URL http://arxiv.org/abs/ 1608.00182 . Peng Tang, Xinggang Wang, Zilong Huang, Xiang Bai, and Wenyu Liu. Deep patch learning for weakly supervised object classification and discovery. CoRR , abs/1705.02429, 2017. URL http://arxiv.org/abs/1705.02429 . Yunchao Wei, Wei Xia, Min Lin, Junshi Huang, Bingbing Ni, Jian Dong, Yao Zhao, and Shuicheng Yan. Hcp: A flexible cnn framework for multi-label image classification. IEEE Trans. Pattern Anal. Mach. Intell. , 38, 2016. Tianjun Xiao, Yichong Xu, Kuiyuan Yang, Jiaxing Zhang, Yuxin Peng, and Zheng Zhang. The application of two-level attention models in deep convolutional neural network for fine-grained image classification. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015 , pp. 842–850, 2015. doi: 10.1109/CVPR.2015.7298685. URLhttps://doi.org/10.1109/CVPR.2015.7298685 . Jianguo Zhang, Marcin Marszalek, Svetlana Lazebnik, and Cordelia Schmid. Local features and kernels for classification of texture and object categories: A comprehensive study. International Journal of Computer Vision , 73(2):213–238, 2007. Bolei Zhou, Aditya Khosla, Àgata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. CoRR , abs/1512.04150, 2015. URL http://arxiv. org/abs/1512.04150 . Luisa M. Zintgraf, Taco S. Cohen, Tameem Adel, and Max Welling. Visualizing deep neural network decisions: Prediction difference analysis. CoRR , abs/1702.04595, 2017. 11Published as a conference paper at ICLR 2019 A A PPENDIX The architecture of the BagNets is detailed in Figure A.1. Training of the models was performed in PyTorch using the default ImageNet training script of Torchvision ( https://github.com/ pytorch/vision , commit 8a4786a) with default parameters. In brief, we used SGD with mo- mentum (0.9), a batchsize of 256 and an initial learning rate of 0:01which we decreased by a factor of 10 every 30 epochs. Images were resized to 256 pixels (shortest side) after which we extracted a random crop of size 224 224 pixels. 3x3 conv, 64 1x1 conv, 64 3x3 conv, 64 1x1 conv, 64 1x1 conv, 64 1x1 conv, 64 1x1 conv, 64 1x1 conv, 641x1 conv, 64 1x1 conv, 64 1x1 conv, 1283x3 conv, 128 1x1 conv, 128 1x1 conv, 1281x1 conv, 128 1x1 conv, 128 1x1 conv, 1281x1 conv, 128 1x1 conv, 128 1x1 conv, 2561x1 conv, 256 1x1 conv, 256 1x1 conv, 2561x1 conv, 256 1x1 conv, 256 1x1 conv, 2561x1 conv, 256 1x1 conv, 256 1x1 conv, 5121x1 conv, 512 1x1 conv, 512 1x1 conv, 5121x1 conv, 512 1x1 conv, 512 1x1 conv, 5121x1 conv, 512 1x1 conv, 512 fc, 1000global avg pool 3 blocks (downsampling only in /f_irst block)4 blocks (downsampling only in /f_irst block)6 blocks (downsampling only in /f_irst block)3 blocks (downsampling only in /f_irst block)3x3 conv, 64 1x1 conv, 64 3x3 conv, 64 1x1 conv, 64 1x1 conv, 64 1x1 conv, 64 1x1 conv, 64 1x1 conv, 641x1 conv, 64 1x1 conv, 64 1x1 conv, 128 3x3 conv, 128 1x1 conv, 128 1x1 conv, 128 1x1 conv, 128 1x1 conv, 128 1x1 conv, 1281x1 conv, 128 1x1 conv, 128 1x1 conv, 2563x3 conv, 256 1x1 conv, 256 1x1 conv, 2561x1 conv, 256 1x1 conv, 256 1x1 conv, 2561x1 conv, 256 1x1 conv, 256 1x1 conv, 5121x1 conv, 512 1x1 conv, 512 1x1 conv, 5121x1 conv, 512 1x1 conv, 512 1x1 conv, 5121x1 conv, 512 1x1 conv, 512 fc, 1000global avg pool3x3 conv, 64 1x1 conv, 64 3x3 conv, 64 1x1 conv, 64 1x1 conv, 64 1x1 conv, 64 1x1 conv, 64 1x1 conv, 641x1 conv, 64 1x1 conv, 64 1x1 conv, 1283x3 conv, 128 1x1 conv, 128 1x1 conv, 1281x1 conv, 128 1x1 conv, 128 1x1 conv, 1281x1 conv, 128 1x1 conv, 128 1x1 conv, 2563x3 conv, 256 1x1 conv, 256 1x1 conv, 2561x1 conv, 256 1x1 conv, 256 1x1 conv, 2561x1 conv, 256 1x1 conv, 256 1x1 conv, 5123x3 conv, 512 1x1 conv, 512 1x1 conv, 5121x1 conv, 512 1x1 conv, 512 1x1 conv, 5121x1 conv, 512 1x1 conv, 512 fc, 1000global avg poolBagNet-33 BagNet-17 BagNet-9 Figure A.1: The BagNet architecture is almost equivalent to the ResNet-50 architectures except for a few changes in the strides and the replacement of most 3 3 convolutions with 1 1 convolutions. Each ResNet block has an expansion of size four (that means the number of output feature maps is four times the number of feature maps within the block). The downsampling operation (dashed arrows) is a simple 1 1 convolution with stride 2. 12Published as a conference paper at ICLR 2019 Original Saliency maps Int. Gradients Epsilon-LRP DeepLIFT (rescale) BagNet-9 BagNet-17 BagNet-33 Figure A.2: Feature attributions of VGG generated using different methods (Saliency, Integrated Gradients,-LRP and DeepLIFT) and feature attributions of BagNets. 13Published as a conference paper at ICLR 2019 BagNet-33 BagNet-17 BagNet-9slot saltshaker vulturestage cello piggy bankBernese mountain artichoke mittencanoe medicine chest hamperLabrador retriever monarch Japanese spanielbanded gecko mortar prisonwreck cassette triumphal archmosquito net flatworm barn spidercocktail shaker stole platesoccer ball umbrella typewriter keyboardpaintbrush pelican amphibianradio hen-of-the-woods cowboy hatbannister birdhouse refrigeratorwater bottle microphone wigsloth bear lifeboat lighterdome vase cabplastic bag hare bolo tie Figure A.3: Same as Figure 3 but for more classes. 14Published as a conference paper at ICLR 2019 A.1 E FFECT OF LOGIT THRESHOLDING We tested how sensitive the classification accuracy of BagNet-33 is with respect to the exact values of the logits for each patch. To this end we thresholded the logits in two ways: first, by setting all values below the threshold to the threshold (and all values above the threshold stay as is). In the second case we binarized the heatmaps by setting all values below the threshold to zero and all values above the threshold to one (this completely removes the amplitude). The results can be found in Figure A.4. Most interestingly, for certain binarization thresholds the top-5 accuracy is within 3-4% of the vanilla BagNet performance. This indicates that the amplitude of the heatmaps is not decisive. Figure A.4: Effect of thresholding the logits on the model performance (top-5 accuracy). 15

You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you s... You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you select really corresponds to the excerpt: there will be details mentioned in the excerpt that should appear in the paper. If you read an abstract and it seems like it could be the paper we’re looking for, read the paper to make sure. Also: sometimes you’ll read a paper that cites the paper we’re looking for. In such cases, please go to the references in order to find the full name of the paper we’re looking for, and search for it, and then select it. Even if we assume fixed filters using a combination of the above, our probabilistic formulation still allows learning the parameters of the GSM experts from data as outlined below. Consequently, we do not need to tune the trade-off weights between the brightness and gradient constancy terms by hand as in [CITATION]. You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you select really corresponds to the excerpt: there will be details mentioned in the excerpt that should appear in the paper. If you read an abstract and it seems like it could be the paper we’re looking for, read the paper to make sure. Also: sometimes you’ll read a paper that cites the paper we’re looking for. In such cases, please go to the references in order to find the full name of the paper we’re looking for, and search for it, and then select it. Even if we assume fixed filters using a combination of the above, our probabilistic formulation still allows learning the parameters of the GSM experts from data as outlined below. Consequently, we do not need to tune the trade-off weights between the brightness and gradient constancy terms by hand as in [CITATION].

- Paper ID: 91228e00fe33ed6072cfe849ab9e98160461549d Title: High Accuracy Optical Flow Estimation Based on a Theory for Warping Abstra... - Paper ID: 91228e00fe33ed6072cfe849ab9e98160461549d Title: High Accuracy Optical Flow Estimation Based on a Theory for Warping Abstract: None Citation Count: 2897 - Paper ID: 9917c7353ebea5466576ab04574a6acd815bc0ce Title: CLG Method for Optical Flow Estimation Based on Gradient Constancy Assumption Abstract: None Citation Count: 4 - Paper ID: 69172d55d9ba2175370bdf35f53fa57bccbf4a7d Title: Improved Accuracy in Gradient-Based Optical Flow Estimation Abstract: None Citation Count: 44 - Paper ID: 95e77b20aad5d8fb3da25a73a7fbbb2b5b5c80aa Title: High Accuracy Optical Flow Method Based on a Theory for Warping: Implementation and Qualitative/Quantitative Evaluation Abstract: None Citation Count: 10 - Paper ID: e93e4ecc7f3acedd7fec150d4eb98f880451cfc3 Title: Optical Flow in Log-Mapped Image Plane-A New Approach Abstract: In this article we propose a novel approach to compute the optical flow directly on log-mapped images. We propose the use of a ... - Paper ID: 91228e00fe33ed6072cfe849ab9e98160461549d Title: High Accuracy Optical Flow Estimation Based on a Theory for Warping Abstract: None Citation Count: 2897 - Paper ID: 9917c7353ebea5466576ab04574a6acd815bc0ce Title: CLG Method for Optical Flow Estimation Based on Gradient Constancy Assumption Abstract: None Citation Count: 4 - Paper ID: 69172d55d9ba2175370bdf35f53fa57bccbf4a7d Title: Improved Accuracy in Gradient-Based Optical Flow Estimation Abstract: None Citation Count: 44 - Paper ID: 95e77b20aad5d8fb3da25a73a7fbbb2b5b5c80aa Title: High Accuracy Optical Flow Method Based on a Theory for Warping: Implementation and Qualitative/Quantitative Evaluation Abstract: None Citation Count: 10 - Paper ID: e93e4ecc7f3acedd7fec150d4eb98f880451cfc3 Title: Optical Flow in Log-Mapped Image Plane-A New Approach Abstract: In this article we propose a novel approach to compute the optical flow directly on log-mapped images. We propose the use of a generalized dynamic image model (GDIM) based method for computing the optical flow as opposed to the brightness constancy model (BCM) based method. We introduce a new notion of "variable window" and use the space-variant form of gradient operator while computing the spatio-temporal gradient in log-mapped images for a better accuracy and to ensure that the local neighborhood is preserved. We emphasize that the proposed method must be numerically accurate, provides a consistent interpretation and is capable of computing the peripheral motion. Experimental results on both the synthetic and real images have been presented to show the efficacy of the proposed method. Citation Count: 18 - Paper ID: 5adcad48c040a13d895ddf4b5778086842cff27d Title: Towards a Multi-camera Generalization of Brightness Constancy Abstract: None Citation Count: 24 - Paper ID: 1a76f1ceb472f96a4963a550da78e5f8d577c268 Title: Optical Flow in Log-mapped Image Plane (A New Approach) Abstract: None Citation Count: 3 - Paper ID: fd2936bc118db8118a9ce04d1c1cc60457764eac Title: An accurate and adaptive optical flow estimation algorithm Abstract: In this paper we present a very accurate algorithm for computing optical flow with non-uniform brightness variations. The proposed algorithm is based on a generalized dynamic image model (GDIM) in conjunction with a regularization framework to cope with the problem of non-uniform brightness variations. To alleviate flow constraint errors due to image aliasing and noise, we employ a reweighted least-squares method to suppress unreliable flow constraints, thus leading to robust estimation of optical flow. In addition, a dynamic smoothness adjustment scheme is proposed to efficiently suppress the smoothness constraint in the vicinity of the motion and brightness variation discontinuities, thus preserving motion boundaries. To efficiently minimize the resulting energy function for optical flow computation, we apply an incomplete Cholesky preconditioned conjugate gradient algorithm to solve the large linear system. Experimental results on some synthetic and real image sequences show that the proposed algorithm outperforms most existing techniques reported in literature in terms of accuracy in optical flow computation with 100 % density. Citation Count: 0 - Paper ID: 7299e15640dd8c26d128322b870e5afd30b98cec Title: Range Flow for Varying Illumination Abstract: None Citation Count: 12

There was an error reading the PDF. Please try a different paper. There was an error reading the PDF. Please try a different paper. There was an error reading the PDF. Please try a different paper.

There was an error reading the PDF. Please try a different paper. There was an error reading the PDF. Please try a different paper. There was an error reading the PDF. Please try a different paper.

High Accuracy Optical Flow Estimation Based on a Theory for Warping⋆ Thomas Brox, Andr´ es Bruhn, Nils Papenberg, and Joachim Weickert Mathe... High Accuracy Optical Flow Estimation Based on a Theory for Warping⋆ Thomas Brox, Andr´ es Bruhn, Nils Papenberg, and Joachim Weickert Mathematical Image Analysis Group Faculty of Mathematics and Computer Science Saarland University, Building 27, 66041 Saarbr¨ ucken, Germany {brox,bruhn,papenberg,weickert }@mia.uni-saarland.de http://www.mia.uni-saarland.de Abstract. We study an energy functional for computing optical flow that com- bines three assumptions: a brightness constancy assumption, a gradient con- stancy assumption, and a discontinuity-preserving spatio-temporal smoothness constraint. In order to allow for large displacements, linearisations in the two data terms are strictly avoided. We present a consistent numerical scheme based on two nested fixed point iterations. By proving that this scheme implements a coarse-to- fine warping strategy, we give a theoretical foundation for warping which has been used on a mainly experimental basis so far. Our evaluation demonstrates that th... High Accuracy Optical Flow Estimation Based on a Theory for Warping⋆ Thomas Brox, Andr´ es Bruhn, Nils Papenberg, and Joachim Weickert Mathematical Image Analysis Group Faculty of Mathematics and Computer Science Saarland University, Building 27, 66041 Saarbr¨ ucken, Germany {brox,bruhn,papenberg,weickert }@mia.uni-saarland.de http://www.mia.uni-saarland.de Abstract. We study an energy functional for computing optical flow that com- bines three assumptions: a brightness constancy assumption, a gradient con- stancy assumption, and a discontinuity-preserving spatio-temporal smoothness constraint. In order to allow for large displacements, linearisations in the two data terms are strictly avoided. We present a consistent numerical scheme based on two nested fixed point iterations. By proving that this scheme implements a coarse-to- fine warping strategy, we give a theoretical foundation for warping which has been used on a mainly experimental basis so far. Our evaluation demonstrates that the novel method gives significantly smaller angular errors than previous techniques for optical flow estimation. We show that it is fairly insensitive to parameter vari- ations, and we demonstrate its excellent robustness under noise. 1 Introduction Optical flow estimation is still one of the key problems in computer vision. Estimating the displacement field between two images, it is applied as soon as correspondences between pixels are needed. Problems of this type are not only restricted to motion estimation, they are also present in a similar fashion in 3D reconstruction or image registration. In the last two decades the quality of optical flow estimation methods has increased dramatically. Starting from the original approaches of Horn and Schunck [11] as well as Lucas and Kanade [15], research developed many new concepts for dealing with shortcomings of previous models. In order to handle discontinuities in the flow field, the quadratic regulariser in the Horn and Schunck model was replaced by smoothness constraints that permit piecewise smooth results [1,9,19,21,25]. Some of these ideas are close in spirit to methods for joint motion estimation and motion segmentation [10,17], and to optical flow methods motivated from robust statistics where outliers are penalised less severely [6,7]. Coarse-to-fine strategies [3,7,16] as well as non-linearised models [19,2] have been used to tackle large displacements. Finally, spatio-temporal approaches have ameliorated the results simply by using the information of an additional dimension [18,6,26,10]. However, not only new ideas have improved the quality of optical flow estimation techniques.Also efforts to obtain a better understanding of what the methods do in detail, ⋆We gratefully acknowledge partial funding by the Deutsche Forschungsgemeinschaft (DFG) . T. Pajdla and J. Matas (Eds.): ECCV 2004, LNCS 3024, pp. 25–36, 2004. c/circlecopyrtSpringer-Verlag Berlin Heidelberg 200426 T. Brox et al. and which effects are caused by changing their parameters, gave an insight into how several models could work together. Furthermore, variational formulations of models gave access to the long experience of numerical mathematics in solving partly difficult optimisation problems. Finding the optimal solution to a certain model is often not trivial, and often the full potential of a model is not used because concessions to implementation aspects have to be made. In this paper we propose a novel variational approach that integrates several of the before mentioned concepts and which can be minimised with a solid numerical method. It is further shown that a coarse-to-fine strategy using the so-called warping technique [7,16], implements the non-linearised optical flow constraint used in [19,2] and in image registration. This has two important effects: Firstly, it becomes possible to integrate the warping technique, which was so far only algorithmically motivated, into a variational framework. Secondly, it shows a theoretically sound way of how image correspondence problems can be solved with an efficient multi-resolution technique. It should be noted that – apart from a very nice paper by Lef´ ebure and Cohen [14] – not many theoretical results on warping are available so far. Finally, the grey value constancy assumption, which is the basic assumption in optical flow estimation, is extended by a gradient constancy assumption. This makes the method robust against grey value changes. While gradient constancy assumptions have also been proposed in [23,22] in order to deal with the aperture problem in the scope of a local approach, their use within variational methods is novel. The experimental evaluation shows that our method yields excellent results. Com- pared to those in the literature, their accuracy is always significantly higher, sometimes even twice as high as the best value known so far. Moreover, the method proved also to be robust under a considerable amount of noise and computation times of only a few seconds per frame on contemporary hardware are possible. Paper Organisation. In the next section, our variational model is introduced, first by discussing all model assumptions, and then in form of an energy based formulation. Section 3 derives a minimisation scheme for this energy. The theoretical foundation of warping methods as a numerical approximation step is given in Section 4. An experi- mental evaluation is presented in Section 5, followed by a brief summary in Section 6. 2 The Variational Model Before deriving a variational formulation for our optical flow method, we give an intuitive idea of which constraints in our view should be included in such a model. – Grey Value Constancy Assumption Since the beginning of optical flow estimation, it has been assumed that the grey value of a pixel is not changed by the displacement. I(x, y, t )=I(x+u, y +v,t +1 ) (1) HereI:Ω⊂R3→Rdenotes a rectangular image sequence, and w:= (u, v, 1)/latticetop is the searched displacement vector between an image at time tand another image at time t+1. The linearised version of the grey value constancy assumption yieldsHigh Accuracy Optical Flow Estimation Based on a Theory for Warping 27 the famous optical flow constraint [11] Ixu+Iyv+It=0 (2) where subscripts denote partial derivatives. However, this linearisation is only valid under the assumption that the image changes linearly along the displacement, which is in general not the case, especially for large displacements. Therefore, our model will use the original, non-linearised grey value constancy assumption (1). – Gradient Constancy Assumption The grey value constancy assumption has one decisive drawback: It is quite suscep- tible to slight changes in brightness, which often appear in natural scenes. Therefore, it is useful to allow some small variations in the grey value and help to determine the displacement vector by a criterion that is invariant under grey value changes. Such a criterion is the gradient of the image grey value, which can also be assumed not to vary due to the displacement [23]. This gives ∇I(x, y, t )=∇I(x+u, y +v,t +1 ). (3) Here∇=(∂x,∂y)/latticetopdenotes the spatial gradient. Again it can be useful to refrain from a linearisation. The constraint (3) is particularly helpful for translatory motion, while constraint (2) can be better suited for more complicated motion patterns. – Smoothness Assumption So far, the model estimates the displacement of a pixel only locally without taking any interaction between neighbouring pixels into account. Therefore, it runs into problems as soon as the gradient vanishes somewhere, or if only the flow in normal direction to the gradient can be estimated ( aperture problem ). Furthermore, one would expect some outliers in the estimates. Hence, it is useful to introduce as a further assumption the smoothness of the flow field. This smoothness constraint can either be applied solely to the spatial domain, if there are only two frames available, or to the spatio-temporal domain, if the displacements in a sequence of images are wanted.As the optimal displacement field will have discontinuities at the boundaries of objects in the scene, it is sensible to generalise the smoothness assumption by demanding a piecewise smooth flow field. – Multiscale Approach In the case of displacements that are larger than one pixel per frame, the cost func- tional in a variational formulation must be expected to be multi-modal, i.e. a min- imisation algorithm could easily be trapped in a local minimum. In order to find the global minimum, it can be useful to apply multiscale ideas: One starts with solving a coarse, smoothed version of the problem by working on the smoothed image sequence. The new problem may have a unique minimum, hopefully close to the global minimum of the original problem. The coarse solution is used as ini- tialisation for solving a refined version of the problem until step by step the original problem is solved. Instead of smoothing the image sequence, it is more efficient to downsample the images respecting the sampling theorem, so the model ends up in a multiresolution strategy. With this description, it is straightforward to derive an energy functional that pe- nalises deviations from these model assumptions. Let x:= (x, y, t )/latticetopand w:=28 T. Brox et al. (u, v, 1)/latticetop. Then the global deviations from the grey value constancy assumption and the gradient constancy assumption are measured by the energy EData (u, v )=/integraldisplay Ω/parenleftbig |I(x+w)−I(x)|2+γ|∇I(x+w)−∇I(x)|2/parenrightbig dx (4) withγbeing a weight between both assumptions. Since with quadratic penalisers, out- liers get too much influence on the estimation, an increasing concave function Ψ(s2)is applied, leading to a robust energy [7,16]: EData (u, v )=/integraldisplay ΩΨ/parenleftbig |I(x+w)−I(x)|2+γ|∇I(x+w)−∇I(x)|2/parenrightbig dx (5) The function Ψcan also be applied separately to each of these two terms. We use the function Ψ(s2)=√ s2+/epsilon12which results in (modified) L1minimisation. Due to the small positive constant /epsilon1,Ψ(s)is still convex which offers advantages in the minimisation process. Moreover, this choice of Ψdoes not introduce any additional parameters, since /epsilon1is only for numerical reasons and can be set to a fixed value, which we choose to be 0.001. Finally, a smoothness term has to describe the model assumption of a piecewise smooth flow field. This is achieved by penalising the total variation of the flow field [20,8], which can be expressed as ESmooth (u, v )=/integraldisplay ΩΨ/parenleftbig |∇3u|2+|∇3v|2/parenrightbig dx. (6) with the same function for Ψas above. The spatio-temporal gradient ∇3:= (∂x,∂y,∂t)/latticetop indicates that a spatio-temporal smoothness assumption is involved. For applications with only two images available it is replaced by the spatial gradient. The total energy is the weighted sum between the data term and the smoothness term E(u, v )=EData +αESmooth (7) with some regularisation parameter α> 0. Now the goal is to find the functions uand vthat minimise this energy. 3 Minimisation 3.1 Euler–Lagrange Equations SinceE(u, v )is highly nonlinear, the minimisation is not trivial. For better readability we define the following abbreviations, where the use of zinstead of temphasises that the expression is nota temporal derivative but a difference that is sought to be minimised. Ix:=∂xI(x+w), Iy:=∂yI(x+w), Iz:=I(x+w)−I(x), Ixx:=∂xxI(x+w), Ixy:=∂xyI(x+w), Iyy:=∂yyI(x+w), Ixz:=∂xI(x+w)−∂xI(x), Iyz:=∂yI(x+w)−∂yI(x).(8)High Accuracy Optical Flow Estimation Based on a Theory for Warping 29 According to the calculus of variations, a minimiser of (7) must fulfill the Euler-Lagrange equations Ψ/prime(I2 z+γ(I2 xz+I2 yz))·(IxIz+γ(IxxIxz+IxyIyz)) −αdiv/parenleftbig Ψ/prime(|∇3u|2+|∇3v|2)∇3u/parenrightbig =0, Ψ/prime(I2 z+γ(I2 xz+I2 yz))·(IyIz+γ(IyyIyz+IxyIxz)) −αdiv/parenleftbig Ψ/prime(|∇3u|2+|∇3v|2)∇3v/parenrightbig =0 with reflecting boundary conditions. 3.2 Numerical Approximation The preceding Euler-Lagrange equations are nonlinear in their argument w=(u, v, 1)/latticetop. A first step towards a linear system of equations, which can be solved with common numerical methods, is the use of fixed point iterations on w. In order to implement a multiscale approach, necessary to better approximate the global optimum of the energy, these fixed point iterations are combined with a downsampling strategy. Instead of the standard downsampling factor of 0.5on each level, it is proposed here to use an arbitrary factorη∈(0,1), what allows smoother transitions from one scale to the next1. Moreover, the full pyramid of images is used, starting with the smallest possible image at the coarsest grid. Let wk=(uk,vk,1)/latticetop,k=0,1,..., with the initialisation w0=( 0,0,1)/latticetopat the coarsest grid. Further, let Ik ∗be the abbreviations defined in (8) but with the iteration variable wkinstead of w. Then wk+1will be the solution of Ψ/prime((Ik+1 z )2+γ((Ik+1 xz )2+(Ik+1 yz )2))·(Ik xIk+1 z +γ(Ik xxIk+1 xz +Ik xyIk+1 yz )) −αdiv/parenleftbig Ψ/prime(|∇3uk+1|2+|∇3vk+1|2)∇3uk+1/parenrightbig =0 Ψ/prime((Ik+1 z )2+γ((Ik+1 xz )2+(Ik+1 yz )2))·(Ik yIk+1 z +γ(Ik yyIk+1 yz +Ik xyIk+1 xz )) −αdiv/parenleftbig Ψ/prime(|∇3uk+1|2+|∇3vk+1|2)∇3vk+1/parenrightbig =0.(9) As soon as a fixed point in wkis reached, we change to the next finer scale and use this solution as initialisation for the fixed point iteration on this scale. Notice that we have a fully implicit scheme for the smoothness term and a semi-implicit scheme for the data term. Implicit schemes are used to yield higher stability and faster convergence. However, this new system is still nonlinear because of the nonlinear func- tionΨ/primeand the symbols Ik+1 ∗. In order to remove the nonlinearity in Ik+1 ∗, first order Taylor expansions are used: Ik+1 z≈Ik z+Ik xduk+Ik ydvk, Ik+1 xz≈Ik xz+Ik xxduk+Ik xydvk, Ik+1 yz≈Ik yz+Ik xyduk+Ik yydvk, where uk+1=uk+dukandvk+1=vk+dvk. So we split the unknowns uk+1,vk+1 in the solutions of the previous iteration step uk,vkand unknown increments duk,d vk. 1Since the grid size in both x- and y-direction is reduced by η, the image size in fact shrinks with a factor η2at each scale.30 T. Brox et al. For better readability let (Ψ/prime)k Data :=Ψ/prime/parenleftBig (Ik z+Ik xduk+Ik ydvk)2 +γ/parenleftbig (Ik xz+Ik xxduk+Ik xydvk)2+(Ik yz+Ik xyduk+Ik yydvk)2/parenrightbig/parenrightBig , (Ψ/prime)k Smooth :=Ψ/prime(|∇3(uk+duk)|2+|∇3(vk+dvk)|2), (10) where (Ψ/prime)k Data can be interpreted as a robustness factor in the data term, and (Ψ/prime)k Smooth as a diffusivity in the smoothness term. With this the first equation in system (9) can be written as 0=(Ψ/prime)k Data·/parenleftBig Ik x/parenleftbig Ik z+Ik xduk+Ik ydvk/parenrightbig/parenrightBig +γ(Ψ/prime)k Data·/parenleftBig Ik xx(Ik xz+Ik xxduk+Ik xydvk)+Ik xy(Ik yz+Ik xyduk+Ik yydvk)/parenrightBig −αdiv/parenleftbig (Ψ/prime)k Smooth ∇3(uk+duk)/parenrightbig , (11) and the second equation can be expressed in a similar way. This is still a nonlinear system of equations for a fixed k, but now in the unknown increments duk,d vk. As the only remaining nonlinearity is due to Ψ/prime, andΨhas been chosen to be a convex function, the remaining optimisation problem is a convex problem, i.e. there exists a unique minimum solution. In order to remove the remaining nonlinearity in Ψ/prime, a second, inner, fixed point iteration loop is applied. Let duk,0:= 0 ,dvk,0:= 0 be our initialisation and let duk,l,d vk,l denote the iteration variables at some step l. Furthermore, let (Ψ/prime)k,l Data and (Ψ/prime)k,l Smooth denote the robustness factor and the diffusivity defined in (10) at iteration k,l. Then finally the linear system of equations in duk,l+1,d vk,l+1reads 0=(Ψ/prime)k,l Data·/parenleftBig Ik x/parenleftbig Ik z+Ik xduk,l+1+Ik ydvk,l+1/parenrightbig +γIk xx(Ik xz+Ik xxduk,l+1+Ik xydvk,l+1)+γIk xy(Ik yz+Ik xyduk,l+1+Ik yydvk,l+1)/parenrightBig −αdiv/parenleftBig (Ψ/prime)k,l Smooth∇3(uk+duk,l+1)/parenrightBig (12) for the first equation. Using standard discretisations for the derivatives, the resulting sparse linear system of equations can now be solved with common numerical methods, such as Gauss-Seidel or SOR iterations. Expressions of type I(x+wk)are computed by means of bilinear interpolation. 4 Relation to Warping Methods Coarse-to-fine warping techniques are a frequently used tool for improving the per- formance of optic flow methods [3,7,17]. While they are often introduced on a purely experimental basis, we show in this section that they can be theoretically justified as a numerical approximation.High Accuracy Optical Flow Estimation Based on a Theory for Warping 31 In order to establish this relation, we restrict ourselves to the grey value constancy model by setting γ=0. Let us also simplify the model by assuming solely spatial smoothness, as in [17]. Under these conditions, (11) can be written as (Ψ/prime)k Data∇Ik(∇Ik)/latticetop/parenleftbigg duk dvk/parenrightbigg −α/parenleftbigg div/parenleftbig (Ψ/prime)k Smooth ∇(uk+duk)/parenrightbig div/parenleftbig (Ψ/prime)k Smooth ∇(vk+dvk)/parenrightbig/parenrightbigg =−(Ψ/prime)k DataIk z∇Ik(13) For a fixed k, this system is equivalent to the Euler–Lagrange equations described in [17]. Also there, only the increments duanddvbetween the first image and the warped second image are estimated. The same increments appear in the outer fixed point iterations of our approach in order to resolve the nonlinearity of the grey value constancy assumption. This shows that the warping technique implements the minimisation of a non-linearised constancy assumption by means of fixed point iterations on w. In earlier approaches, the main motivation for warping has been the coarse-to-fine strategy. Due to solutions uandvcomputed on coarser grids, only an increment duand dvhad to be computed on the fine grid. Thus, the estimates used to have a magnitude of less than one pixel per frame, independent of the magnitude of the total displacement. This ability to deal with larger displacements proved to be a very important aspect in differential optical flow estimation. A second strategy to deal with large displacements has been the usage of the non- linearised grey value constancy assumption [19,2]. Here, large displacements are allowed from the beginning. However, the nonlinearity results in a multi-modal functional. In such a setting, the coarse-to-fine strategy is not only wanted, but even necessary to better approximate the global minimum. At the end, both strategies not only lead to similar results. In fact, as we have seen above, they are completely equivalent.As a consequence, the coarse-to-fine warping technique can be formulated as a single minimisation problem, and image registration techniques relying on non-linearised constancy assumptions get access to an efficient multiresolution method for minimising their energy functionals. 5 Evaluation For evaluation purposes experiments with both synthetic and real-world image data were performed. The presented angular errors were computed according to [5]. Let us start our evaluation with the two variants of a famous sequence: the Yosemite sequence with and without cloudy sky. The original version with cloudy sky was cre- ated by Lynn Quam and is available at ftp://ftp.csd.uwo.ca/pub/vision . It com- bines both divergent and translational motion. The version without clouds is available athttp://www.cs.brown.edu/people/black/images.html . Tab.1 shows a comparison of our results for both sequences to the best results from the literature. As one can see, our variational approach outperforms all other methods. Regarding the sequence with clouds, we achieve results that are more than twice as accurate as all results from the literature. For the sequence without clouds, angular errors below 1 degree are reached for the first time with a method that offers full density. The corresponding flow fields presented in Fig.1 give a qualitative impression of these raw numbers: They match the ground truth very well. Not only the discontinuity between the32 T. Brox et al. Table 1. Comparison between the results from the literature with 100 % density and our results for theYosemite sequence with and without cloudy sky. AAE = average angular error. STD = standard deviation. 2D = spatial smoothness assumption. 3D = spatio-temporal smoothness assumption. Yosemite with clouds Yosemite without clouds Technique AAE STD Technique AAE STD Nagel [5] 10.22◦16.51◦Juet al. [12] 2.16◦2.00◦ Horn–Schunck, mod. [5] 9.78◦16.19◦Bab-Hadiashar–Suter [4] 2.05◦2.92◦ Uras et al. [5] 8.94◦15.61◦Lai–Vemuri [13] 1.99◦1.41◦ Alvarez et al. [2] 5.53◦7.40◦Our method (2D) 1.59◦1.39◦ Weickert et al. [24] 5.18◦8.68◦M´emin–P´ erez [16] 1.58◦1.21◦ M´emin–P´ erez [16] 4.69◦6.89◦Weickert et al. [24] 1.46◦1.50◦ Our method (2D) 2.46◦7.31◦Farneb¨ ack [10] 1.14◦2.14◦ Our method (3D) 1.94◦6.02◦Our method (3D) 0.98◦1.17◦ Table 2. Results for the Yosemite sequence with and without cloudy sky. Gaussian noise with varying standard deviations σnwas added, and the average angular errors and their standard deviations were computed. AAE = average angular error. STD = standard deviation. Yosemite with clouds Yosemite without clouds σn AAE STD σn AAE STD 0 1.94◦6.02◦0 0.98◦1.17◦ 10 2.50◦5.96◦10 1.26◦1.29◦ 20 3.12◦6.24◦20 1.63◦1.39◦ 30 3.77◦6.54◦30 2.03◦1.53◦ 40 4.37◦7.12◦40 2.40◦1.71◦ two types of motion is preserved, also the translational motion of the clouds is estimated accurately. The reason for this behaviour lies in our assumptions, that are clearly stated in the energy functional: While the choice of the smoothness term allows discontinuities, the gradient constancy assumption is able to handle brightness changes – like in the area of the clouds. Because of the presence of second order image derivatives in the Euler-Lagrange equations, we tested the influence of noise on the performance of our method in the next experiment. We added Gaussian noise of mean zero and different standard deviations to both sequences. The obtained results are presented in Tab.2. They show that our approach even yields excellent flow estimates when severe noise is present: For the cloudy Yosemite sequence, our average angular error for noise with standard deviation 40 is better than all results from the literature for the sequence without noise. In a third experiment we evaluated the robustness of the free parameters in our approach: the weight γbetween the grey value and the gradient constancy assumption, and the smoothness parameter α. Often an image sequence is preprocessed by Gaussian convolution with standard deviation σ[5]. In this case, σcan be regarded as a third parameter. We computed results with parameter settings that deviated by a factor 2 in both directions from the optimum setting. The outcome listed in Tab. 3 shows that the method is also very robust under parameter variations. Although our paper does not focus on fast computation but on high accuracy, the implicit minimisation scheme presented here is also reasonably fast, especially if theHigh Accuracy Optical Flow Estimation Based on a Theory for Warping 33 Fig. 1. (a) Top left: Frame 8 of the Yosemite sequence without clouds. (b) Top right: Corresponding frame of the sequence with clouds. (c) Middle left: Ground truth without clouds. (d) Middle right: Ground truth with clouds. (e) Bottom left: Computed flow field by our 3D method for the sequence without clouds. (f) Bottom right: Ditto for the sequence with clouds. reduction factor ηis lowered or if the iterations are stopped before full convergence. The convergence behaviour and computation times can be found in Tab. 4. Computations have been performed on a 3.06 GHz Intel Pentium 4 processor executing C/C++ code. For evaluating the performance of our method for real-world image data, the Ettlinger Tortraffic sequence by Nagel was used. This sequence consists of 50 frames of size34 T. Brox et al. Table 3. Parameter variation for our method with spatio-temporal smoothness assumption. Yosemite with clouds σαγ AAE 0.8 80 100 1.94◦ 0.4 80 100 2.10◦ 1.6 80 100 2.04◦ 0.8 40 100 2.67◦ 0.8 160 100 2.21◦ 0.8 80 50 2.07◦ 0.8 80 200 2.03◦ Table 4. Computation times and convergence for Yosemite sequence with clouds. 3D - spatio-temporal method reduction outer fixed inner fixed SOR computation AAE factor η point iter. point iter. iter. time/frame 0.95 77 5 10 23.4s 1.94◦ 0.90 38 2 10 5.1s 2.09◦ 0.80 18 2 10 2.7s 2.56◦ 0.75 14 1 10 1.2s 3.44◦ Fig. 2. (a) Left: Computed flow field between frame 5 and 6 of the Ettlinger Tor traffic sequence. (b) Right: Computed magnitude of the optical flow field. 512×512. It is available at http://i21www.ira.uka.de/image sequences/ . In Fig. 2 the computed flow field and its magnitude are shown. Our estimation gives very realistic results, and the algorithm hardly suffers from interlacing artifacts that are present in all frames. Moreover, the flow boundaries are rather sharp and can be used directly for segmentation purposes by applying a simple thresholding step.High Accuracy Optical Flow Estimation Based on a Theory for Warping 35 6 Conclusion In this paper we have investigated a continuous, rotationally invariant energy functional for optical flow computations based on two terms: a robust data term with a bright- ness constancy and a gradient constancy assumption, combined with a discontinuity- preserving spatio-temporal TV regulariser. While each of these concepts has proved its use before (see e.g. [22,26]), we have shown that their combination outperforms all methods from the literature so far. One of the main reasons for this performance is the use of an energy functional with non-linearised data term and our strategy to consequently postpone all linearisations to the numerical scheme: While linearisations in the model immediately compromise the overall performance of the system, linearisations in the numerical scheme can help to improve convergence to the global minimum. Another im- portant result in our paper is the proof that the widely-used warping can be theoretically justified as a numerical approximation strategy that does not influence the continuous model . We hope that this strategy of transparent continuous modelling in conjunction with consistent numerical approximations shows that excellent performance and deeper theoretical understanding are not contradictive: They are nothing else but two sides of the same medal. References 1. L. Alvarez, J. Esclar ´in, M. Lef´ ebure, and J. S´ anchez. A PDE model for computing the optical flow. In Proc. XVI Congreso de Ecuaciones Diferenciales y Aplicaciones , pages 1349–1356, Las Palmas de Gran Canaria, Spain, Sept. 1999. 2. L. Alvarez, J. Weickert, and J. S´ anchez. Reliable estimation of dense optical flow fields with large displacements. International Journal of Computer Vision , 39(1):41–56, Aug. 2000. 3. P. Anandan. A computational framework and an algorithm for the measurement of visual motion. International Journal of Computer Vision , 2:283–310, 1989. 4. A. Bab-Hadiashar and D. Suter. Robust optic flow computation. International Journal of Computer Vision , 29(1):59–77, Aug. 1998. 5. J. L. Barron, D. J. Fleet, and S. S. Beauchemin. Performance of optical flow techniques. International Journal of Computer Vision , 12(1):43–77, Feb. 1994. 6. M. J. Black and P.Anandan. Robust dynamic motion estimation over time. In Proc. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition , pages 292–302, Maui, HI, June 1991. IEEE Computer Society Press. 7. M. J. Black and P. Anandan. The robust estimation of multiple motions: parametric and piecewise smooth flow fields. Computer Vision and Image Understanding , 63(1):75–104, Jan. 1996. 8. I. Cohen. Nonlinear variational method for optical flow computation. In Proc. Eighth Scan- dinavian Conference on Image Analysis , volume 1, pages 523–530, Tromsø, Norway, May 1993. 9. R. Deriche, P. Kornprobst, and G. Aubert. Optical-flow estimation while preserving its dis- continuities: a variational approach. In Proc. Second Asian Conference on Computer Vision , volume 2, pages 290–295, Singapore, Dec. 1995. 10. G. Farneb¨ ack. Very high accuracy velocity estimation using orientation tensors, parametric motion, and simultaneous segmentation of the motion field. In Proc. Eighth International Conference on Computer Vision , volume 1, pages 171–177, Vancouver, Canada, July 2001. IEEE Computer Society Press.36 T. Brox et al. 11. B. Horn and B. Schunck. Determining optical flow. Artificial Intelligence , 17:185–203, 1981. 12. S. Ju, M. Black, and A. Jepson. Skin and bones: multi-layer, locally affine, optical flow and regularization with transparency. In Proc. 1996 IEEE Computer Society Conference on Computer Vision and Pattern Recognition , pages 307–314, San Francisco, CA, June 1996. IEEE Computer Society Press. 13. S.-H. Lai and B. C. Vemuri. Reliable and efficient computation of optical flow. International Journal of Computer Vision , 29(2):87–105, Oct. 1998. 14. M. Lef´ ebure and L. D. Cohen. Image registration, optical flow and local rigidity. Journal of Mathematical Imaging and Vision , 14(2):131–147, Mar. 2001. 15. B. Lucas and T. Kanade. An iterative image registration technique with an application to stereo vision. In Proc. Seventh International Joint Conference on Artificial Intelligence , pages 674–679, Vancouver, Canada, Aug. 1981. 16. E. M´ emin and P. P´ erez. A multigrid approach for hierarchical motion estimation. In Proc. Sixth International Conference on Computer Vision , pages 933–938, Bombay, India, Jan. 1998. Narosa Publishing House. 17. E. M´ emin and P. P´ erez. Hierarchical estimation and segmentation of dense motion fields. International Journal of Computer Vision , 46(2):129–155, 2002. 18. H.-H. Nagel. Extending the ’oriented smoothness constraint’ into the temporal domain and the estimation of derivatives of optical flow. In O. Faugeras, editor, Computer Vision – ECCV ’90, volume 427 of Lecture Notes in Computer Science , pages 139–148. Springer, Berlin, 1990. 19. H.-H. Nagel and W. Enkelmann. An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence , 8:565–593, 1986. 20. L. I. Rudin, S. Osher, and E. Fatemi. Nonlinear total variation based noise removal algorithms. Physica D , 60:259–268, 1992. 21. C. Schn¨ orr. Segmentation of visual motion by minimizing convex non-quadratic functionals. InProc. Twelfth International Conference on Pattern Recognition , volume A, pages 661–663, Jerusalem, Israel, Oct. 1994. IEEE Computer Society Press. 22. M. Tistarelli. Multiple constraints for optical flow. In J.-O. Eklundh, editor, Computer Vision – ECCV ’94 , volume 800 of Lecture Notes in Computer Science , pages 61–70. Springer, Berlin, 1994. 23. S. Uras, F. Girosi, A. Verri, and V . Torre. A computational approach to motion perception. Biological Cybernetics , 60:79–87, 1988. 24. J. Weickert, A. Bruhn, and C. Schn¨ orr. Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods. Technical Report 82, Dept. of Mathematics, Saarland University, Saarbr¨ ucken, Germany, Apr. 2003. 25. J. Weickert and C. Schn¨ orr. A theoretical framework for convex regularizers in PDE-based computation of image motion. International Journal of Computer Vision , 45(3):245–264, Dec. 2001. 26. J. Weickert and C. Schn¨ orr. Variational optic flow computation with a spatio-temporal smooth- ness constraint. Journal of Mathematical Imaging and Vision , 14(3):245–255, May 2001.

You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you s... You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you select really corresponds to the excerpt: there will be details mentioned in the excerpt that should appear in the paper. If you read an abstract and it seems like it could be the paper we’re looking for, read the paper to make sure. Also: sometimes you’ll read a paper that cites the paper we’re looking for. In such cases, please go to the references in order to find the full name of the paper we’re looking for, and search for it, and then select it. We learn π using behavioral cloning [CITATION], which optimizes π by minimizing the negative log-likelihood of actions at given the images and language instructions. You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you select really corresponds to the excerpt: there will be details mentioned in the excerpt that should appear in the paper. If you read an abstract and it seems like it could be the paper we’re looking for, read the paper to make sure. Also: sometimes you’ll read a paper that cites the paper we’re looking for. In such cases, please go to the references in order to find the full name of the paper we’re looking for, and search for it, and then select it. We learn π using behavioral cloning [CITATION], which optimizes π by minimizing the negative log-likelihood of actions at given the images and language instructions.

- Paper ID: e134ec1e834b0d56945916cdc02df653dc4e175d Title: Diffusion Model-Augmented Behavioral Cloning Abstract: Imitation learning ... - Paper ID: e134ec1e834b0d56945916cdc02df653dc4e175d Title: Diffusion Model-Augmented Behavioral Cloning Abstract: Imitation learning addresses the challenge of learning by observing an expert's demonstrations without access to reward signals from environments. Most existing imitation learning methods that do not require interacting with environments either model the expert distribution as the conditional probability p(a|s) (e.g., behavioral cloning, BC) or the joint probability p(s, a). Despite its simplicity, modeling the conditional probability with BC usually struggles with generalization. While modeling the joint probability can lead to improved generalization performance, the inference procedure is often time-consuming and the model can suffer from manifold overfitting. This work proposes an imitation learning framework that benefits from modeling both the conditional and joint probability of the expert distribution. Our proposed diffusion model-augmented behavioral cl... - Paper ID: e134ec1e834b0d56945916cdc02df653dc4e175d Title: Diffusion Model-Augmented Behavioral Cloning Abstract: Imitation learning addresses the challenge of learning by observing an expert's demonstrations without access to reward signals from environments. Most existing imitation learning methods that do not require interacting with environments either model the expert distribution as the conditional probability p(a|s) (e.g., behavioral cloning, BC) or the joint probability p(s, a). Despite its simplicity, modeling the conditional probability with BC usually struggles with generalization. While modeling the joint probability can lead to improved generalization performance, the inference procedure is often time-consuming and the model can suffer from manifold overfitting. This work proposes an imitation learning framework that benefits from modeling both the conditional and joint probability of the expert distribution. Our proposed diffusion model-augmented behavioral cloning (DBC) employs a diffusion model trained to model expert behaviors and learns a policy to optimize both the BC loss (conditional) and our proposed diffusion model loss (joint). DBC outperforms baselines in various continuous control tasks in navigation, robot arm manipulation, dexterous manipulation, and locomotion. We design additional experiments to verify the limitations of modeling either the conditional probability or the joint probability of the expert distribution as well as compare different generative models. Ablation studies justify the effectiveness of our design choices. Citation Count: 12 - Paper ID: e72112748349b8422c0642fbf5679cf97bbe1a52 Title: Driver Behavioral Cloning for Route Following in Autonomous Vehicles Using Task Knowledge Distillation Abstract: Planning appropriate driving trajectory for route following is an important function for autonomous driving. Behavioral cloning, which allows automatic trajectory learning and improvement, has been effectively used in driving trajectory planning. However, existing behavioral cloning methods always rely on large scales of time-consuming, laborious, and reliable labels. To address this problem, this paper proposes a new off-policy imitation learning method for autonomous driving using task knowledge distillation. This novel method clones human driving behavior and effectively transfers the driving strategies to domain shift scenarios. The experiment results indicate that our method can lead to satisfactory route-following performance in realistic urban driving scenes and can transfer the driving strategies to new unknown scenes under various illumination and weather scenarios for autonomous driving. Citation Count: 10 - Paper ID: 5c74b1021aebd0575c339ce4dfd47e183009a9c5 Title: Implicit Behavioral Cloning Abstract: We find that across a wide range of robot policy learning scenarios, treating supervised policy learning with an implicit model generally performs better, on average, than commonly used explicit models. We present extensive experiments on this finding, and we provide both intuitive insight and theoretical arguments distinguishing the properties of implicit models compared to their explicit counterparts, particularly with respect to approximating complex, potentially discontinuous and multi-valued (set-valued) functions. On robotic policy learning tasks we show that implicit behavioral cloning policies with energy-based models (EBM) often outperform common explicit (Mean Square Error, or Mixture Density) behavioral cloning policies, including on tasks with high-dimensional action spaces and visual image inputs. We find these policies provide competitive results or outperform state-of-the-art offline reinforcement learning methods on the challenging human-expert tasks from the D4RL benchmark suite, despite using no reward information. In the real world, robots with implicit policies can learn complex and remarkably subtle behaviors on contact-rich tasks from human demonstrations, including tasks with high combinatorial complexity and tasks requiring 1mm precision. Citation Count: 219 - Paper ID: 35da1cd669ad5492a6358ea53aea95de28d39ded Title: Behavioral Cloning from Observation Abstract: Humans often learn how to perform tasks via imitation: they observe others perform a task, and then very quickly infer the appropriate actions to take based on their observations. While extending this paradigm to autonomous agents is a well-studied problem in general, there are two particular aspects that have largely been overlooked: (1) that the learning is done from observation only (i.e., without explicit action information), and (2) that the learning is typically done very quickly. In this work, we propose a two-phase, autonomous imitation learning technique called behavioral cloning from observation (BCO), that aims to provide improved performance with respect to both of these aspects. First, we allow the agent to acquire experience in a self-supervised fashion. This experience is used to develop a model which is then utilized to learn a particular task by observing an expert perform that task without the knowledge of the specific actions taken. We experimentally compare BCO to imitation learning methods, including the state-of-the-art, generative adversarial imitation learning (GAIL) technique, and we show comparable task performance in several different simulation domains while exhibiting increased learning speed after expert trajectories become available. Citation Count: 539 - Paper ID: 3fa5cfd3745b1b4df814283eac955a560e746bfd Title: When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning? Abstract: Offline reinforcement learning (RL) algorithms can acquire effective policies by utilizing previously collected experience, without any online interaction. It is widely understood that offline RL is able to extract good policies even from highly suboptimal data, a scenario where imitation learning finds suboptimal solutions that do not improve over the demonstrator that generated the dataset. However, another common use case for practitioners is to learn from data that resembles demonstrations. In this case, one can choose to apply offline RL, but can also use behavioral cloning (BC) algorithms, which mimic a subset of the dataset via supervised learning. Therefore, it seems natural to ask: when can an offline RL method outperform BC with an equal amount of expert data, even when BC is a natural choice? To answer this question, we characterize the properties of environments that allow offline RL methods to perform better than BC methods, even when only provided with expert data. Additionally, we show that policies trained on sufficiently noisy suboptimal data can attain better performance than even BC algorithms with expert data, especially on long-horizon problems. We validate our theoretical results via extensive experiments on both diagnostic and high-dimensional domains including robotic manipulation, maze navigation, and Atari games, with a variety of data distributions. We observe that, under specific but common conditions such as sparse rewards or noisy data sources, modern offline RL methods can significantly outperform BC. Citation Count: 43 - Paper ID: 26db91bc3f914a7c3048a82946bf4261933940a0 Title: Approaches That Use Domain-Specific Expertise: Behavioral-Cloning-Based Advantage Actor-Critic in Basketball Games Abstract: Research on the application of artificial intelligence (AI) in games has recently gained momentum. Most commercial games still use AI based on a finite state machine (FSM) due to complexity and cost considerations. However, FSM-based AI decreases user satisfaction given that it performs the same patterns of consecutive actions in the same situations. This necessitates a new AI approach that applies domain-specific expertise to existing reinforcement learning algorithms. We propose a behavioral-cloning-based advantage actor-critic (A2C) that improves learning performance by applying a behavioral cloning algorithm to an A2C algorithm in basketball games. The state normalization, reward function, and episode classification approaches are used with the behavioral-cloning-based A2C. The results of the comparative experiments with the traditional A2C algorithms validated the proposed method. Our proposed method using existing approaches solved the difficulty of learning in basketball games. Citation Count: 3 - Paper ID: a8c409791404828a3276ffb0e4a71d3263b0526e Title: Should I Run Offline Reinforcement Learning or Behavioral Cloning? Abstract: Offline reinforcement learning (RL) algorithms can acquire effective policies by utilizing Citation Count: 53 - Paper ID: f6b14bd44ace981b786abc335e209576612ae6f4 Title: Asking for Help: Failure Prediction in Behavioral Cloning through Value Approximation Abstract: Recent progress in end-to-end Imitation Learning approaches has shown promising results and generalization capabilities on mobile manipulation tasks. Such models are seeing increasing deployment in real-world settings, where scaling up requires robots to be able to operate with high autonomy, i.e. requiring as little human supervision as possible. In order to avoid the need for one-on-one human supervision, robots need to be able to detect and prevent policy failures ahead of time, and ask for help, allowing a remote operator to supervise multiple robots and help when needed. However, the black-box nature of end-to-end Imitation Learning models such as Behavioral Cloning, as well as the lack of an explicit state-value representation, make it difficult to predict failures. To this end, we introduce Behavioral Cloning Value Approximation (BCVA), an approach to learning a state value function based on and trained jointly with a Behavioral Cloning policy that can be used to predict failures. We demonstrate the effectiveness of BCVA by applying it to the challenging mobile manipulation task of latched-door opening, showing that we can identify failure scenarios with with 86% precision and 81 % recall, evaluated on over 2000 real world runs, improving upon the baseline of simple failure classification by 10 percentage-points. Citation Count: 1 - Paper ID: 9059ceec2b1d7f53924f81f9f607b4b1006ae018 Title: Distributionally Robust Behavioral Cloning for Robust Imitation Learning Abstract: Robust reinforcement learning (RL) aims to learn a policy that can withstand uncertainties in model parameters, which often arise in practical RL applications due to modeling errors in simulators, variations in real-world system dynamics, and adversarial disturbances. This paper introduces the robust imitation learning (IL) problem in a Markov decision process (MDP) framework where an agent learns to mimic an expert demonstrator that can withstand uncertainties in model parameters without additional online environment interactions. The agent is only provided with a dataset of state-action pairs from the expert on a single (nominal) dynamics, without any information about the true rewards from the environment. Behavioral cloning (BC), a supervised learning method, is a powerful algorithm to address the vanilla IL problem. We propose an algorithm for the robust IL problem that utilizes distributionally robust optimization (DRO) with BC. We call the algorithm DR-BC and show its robust performance against parameter uncertainties both in theory and in practice. We also demonstrate the empirical performance of our approach to addressing model perturbations on several MuJoCo continuous control tasks. Citation Count: 0 - Paper ID: 9f2357749d35d0a721a4f0cbb46c5766426a800e Title: Behavioral Cloning via Search in Embedded Demonstration Dataset Abstract: Behavioural cloning uses a dataset of demonstrations to learn a behavioural policy. To overcome various learning and policy adaptation problems, we propose to use latent space to index a demonstration dataset, instantly access similar relevant experiences, and copy behavior from these situations. Actions from a selected similar situation can be performed by the agent until representations of the agent's current situation and the selected experience diverge in the latent space. Thus, we formulate our control problem as a search problem over a dataset of experts' demonstrations. We test our approach on BASALT MineRL-dataset in the latent representation of a Video PreTraining model. We compare our model to state-of-the-art Minecraft agents. Our approach can effectively recover meaningful demonstrations and show human-like behavior of an agent in the Minecraft environment in a wide variety of scenarios. Experimental results reveal that performance of our search-based approach is comparable to trained models, while allowing zero-shot task adaptation by changing the demonstration examples. Citation Count: 3

Behavioral Cloning from Observation Faraz Torabi1, Garrett Warnell2, Peter Stone1 1The University of Texas at Austin 2U.S. Army Research Lab... Behavioral Cloning from Observation Faraz Torabi1, Garrett Warnell2, Peter Stone1 1The University of Texas at Austin 2U.S. Army Research Laboratory ffaraztrb,pstoneg@cs.utexas.edu, garrett.a.warnell.civ@mail.mil Abstract Humans often learn how to perform tasks via im- itation: they observe others perform a task, and then very quickly infer the appropriate actions to take based on their observations. While extend- ing this paradigm to autonomous agents is a well- studied problem in general, there are two particu- lar aspects that have largely been overlooked: (1) that the learning is done from observation only (i.e., without explicit action information), and (2) that the learning is typically done very quickly. In this work, we propose a two-phase, autonomous imi- tation learning technique called behavioral cloning from observation (BCO), that aims to provide im- proved performance with respect to both of these aspects. First, we allow the agent to acquire ex- perience in a self-supervi... Behavioral Cloning from Observation Faraz Torabi1, Garrett Warnell2, Peter Stone1 1The University of Texas at Austin 2U.S. Army Research Laboratory ffaraztrb,pstoneg@cs.utexas.edu, garrett.a.warnell.civ@mail.mil Abstract Humans often learn how to perform tasks via im- itation: they observe others perform a task, and then very quickly infer the appropriate actions to take based on their observations. While extend- ing this paradigm to autonomous agents is a well- studied problem in general, there are two particu- lar aspects that have largely been overlooked: (1) that the learning is done from observation only (i.e., without explicit action information), and (2) that the learning is typically done very quickly. In this work, we propose a two-phase, autonomous imi- tation learning technique called behavioral cloning from observation (BCO), that aims to provide im- proved performance with respect to both of these aspects. First, we allow the agent to acquire ex- perience in a self-supervised fashion. This expe- rience is used to develop a model which is then utilized to learn a particular task by observing an expert perform that task without the knowledge of the specific actions taken. We experimentally com- pare BCO to imitation learning methods, including the state-of-the-art, generative adversarial imitation learning (GAIL) technique, and we show compara- ble task performance in several different simulation domains while exhibiting increased learning speed after expert trajectories become available. 1 Introduction The ability to learn through experience is a hallmark of intel- ligence. Humans most certainly learn this way, and, using reinforcement learning (RL) [Sutton and Barto, 1998 ], au- tonomous agents may do so as well. However, learning to perform a task based solely on one’s own experience can be very difficult and slow. Humans are often able to learn much faster by observing and imitating the behavior of others. En- abling this same ability in autonomous agents, referred to as learning from demonstration (LfD), has been given a great deal of attention in the research community [Schaal, 1997; Argall et al., 2009 ]. While much of LfD research is motivated by the way hu- mans learn from observing others, it has largely overlooked the integration of two important aspects of that paradigm.Behavioral Cloning from Observation (BCO)START Initialize policyi=0 0  State-only demonstrationsDdemoRun policyi f(sa t;sa t+1)g fatgAppend to Ta ;A Ta ;A Update modelMi Mi  Infer actions~Ademo SdemoUpdate policyi i  Figure 1: Behavioral Cloning from Observation (BCO( )) frame- work proposed in this paper. The agent is initialized with a (random) policy which interacts with the environment and collects data to to learn its own agent-specific inverse dynamics model. Then, given state-only demonstration information, the agent uses this learned model to infer the expert’s missing action information. Once these actions have been inferred, the agent performs imitation learning. The updated policy is then used to collect data and this process re- peats. First, unlike the classical LfD setting, humans do not typically have knowledge of the precise actions executed by demon- strators, i.e., the internal control signals demonstrators use to guide their behavior Second, humans are able to perform im- itation without needing to spend a lot of time interacting with their environment after the demonstration has been provided. Most LfD work is unlike human imitation in its assump- tion that imitators know the actions executed by demonstra- tors. Human imitators typically do not have access to this information, and requiring it immediately precludes using a large amount of demonstration data where action sequences are not given. For example, there is a great number of tutorial videos on YouTube that only provide the observer knowledge of the demonstrator’s state trajectory. It would be immensely beneficial if we could devise LfD algorithms to make use of such information. Another challenge faced by LfD is the necessity of environ- ment interaction, which can be expensive in several regards. One is the amount of time it requires: executing actions, ei- ther in the real world or in simulation, takes time. If a learn- ing algorithm requires that a large number of actions must be executed in order to find a good imitation policy after aProceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18) 4950demonstration is presented, then there will be an undesirable amount of delay before the imitating agent will be successful. Furthermore, algorithms that require post-demonstration in- teraction typically require it again and again for each newly- demonstrated task, which could result in even more delay. Beyond delay, environment interaction can also be risky. For example, when training autonomous vehicles, operating on city streets while learning might endanger lives or lead to costly damage due to crashes. Therefore, we desire an algo- rithm for which environment interactions can be performed as a pre-processing step - perhaps in a safer environment - and where the information learned from those interactions can be re-used for a variety of demonstrated tasks. In this paper, we propose a new imitation learning al- gorithm called behavioral cloning from observation (BCO) . BCO simultaneously addresses both of the issues discussed above, i.e., it provides reasonable imitation policies almost immediately upon observing state-trajectory-only demonstra- tions. First, it calls for the agent to learn a task-independent, inverse dynamics model in a pre-demonstration, exploratory phase. Then, upon observation of a demonstration without action information, BCO uses the learned model to infer the missing actions. Finally, BCO uses the demonstration and the inferred actions to find a policy via behavioral cloning. If post-demonstration environment interaction is allowed, BCO additionally specifies an iterative scheme where the agent uses the extra interaction time in order to learn a better model and improve its imitation policy. This iterative scheme there- fore provides a tradeoff between imitation performance and post-demonstration environment interaction. 2 Related Work BCO is related to both imitation learning and model-based learning. We begin with a review of imitation learning ap- proaches, which typically fall under one of two broad cat- egories: behavioral cloning (BC) andinverse reinforcement learning (IRL). Behavioral cloning [Bain and Sommut, 1999; Ross et al., 2011; Daftry et al., 2016 ]is one of the main methods to ap- proach an imitation learning problem. The agent receives as training data both the encountered states and actions of the demonstrator, and then uses a classifier or regressor to replicate the expert’s policy [Ross and Bagnell, 2010 ]. This method is powerful in the sense that it is capable of imitat- ing the demonstrator immediately without having to interact with the environment. Accordingly, BC has been used in a variety of applications. For instance, it has been used to train a quadrotor to fly down a forest trail [Giusti et al., 2016 ]. There, the training data is the pictures of the forest trail la- beled with the actions that the demonstrating quadrotor used, and the policy is modeled as a convolutional neural network classifier. In the end, the quadrotor manages to fly down the trail successfully. BC has also been used in autonomous driv- ing[Bojarski et al., 2016 ]. The training data is acquired from a human demonstrator, and a convolutional neural network is trained to map raw pixels from a single front-facing cam- era directly to steering commands. After training, the vehicle is capable of driving in traffic on local roads. BC has alsobeen successfully used to teach manipulator robots complex, multi-step, real-world tasks using kinesthetic demonstration [Niekum et al., 2015 ]. While behavioral cloning is powerful, it is also only applicable in scenarios where the demonstra- tor’s action sequences are available. However, when humans want to imitate each other’s actions, they do not have access to the internal control signals the demonstrator used. Instead, they only see the effects of those actions. In our setting, we wish to perform imitation in these scenarios, and so we can- not apply BC techniques as they are typically formulated. Inverse reinforcement learning is a second category of im- itation learning. IRL techniques seek to learn a cost func- tion that has the minimum value for the demonstrated ac- tions. The learned cost function is then used in combina- tion with RL methods to find an imitation policy. Like BC techniques, IRL methods usually assume that state-action pairs are available [Finn et al., 2016; Ho and Ermon, 2016; Hoet al., 2016 ], and also that the reward is a function of both states and actions. An exception is the work of Liu et al.[2017]. In this work, it is assumed that both demonstrator and imitator are capable of following a trajectory at the exact same pace to perform a task, and the IRL method defines the reward signal to be the proximity of the imitator and demon- strator’s encoded state features at each time step. As a result, the reward signal can only be generated after the demonstra- tion is made available, after which reinforcement learning and environment interaction must be completed in order to find a good policy. In our work, we wish to minimize the amount of environment interaction necessary after the demonstration is provided, and so we seek an alternative to IRL. BCO is also related to the model-based learning litera- ture in that it makes use of learned models of the environ- ment. In general, model-based methods have major advan- tages over those that are model-free. First, they are more sample-efficient [Chebotar et al., 2017 ], i.e., they do not re- quire as many environment interactions as model-free meth- ods. Second, the learned models can be transferred across tasks [Taylor et al., 2008 ]. Typical model-learning techniques focus on obtaining an estimate of the transition dynamics model, i.e., a mapping from current state and action to the next state. In our work, on the other hand, we want the agent to learn a model of the environment that will help us infer missing actions, and therefore BCO learns a slightly-different inverse dynamics model, i.e., a mapping from state transitions to the actions [Hanna and Stone, 2017 ]. There has also been recent work done where inverse mod- els have been used to perform imitation learning in the ab- sence of action information. Niekum et al [2015a] present such a technique for situations in which the kinematic equa- tions are known. Nair et al [2017] propose a technique that first learns an inverse dynamics model and then use that model to estimate the missing action information at each time step from a single demonstration. The method we develop here, on the other hand, both does not assume prior knowl- edge of the inverse model and is capable of generalizing in cases when mulitiple demonstrations are available.Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18) 4951BCO(0) BCO( ) GAIL & FEMPre-demonstration Post-demonstration jIprej jIprej jIprejjIprejjIprej...jIprej ...jIIRLj Figure 2: Learning timelines for BCO(0 ), BCO( ), and the IRL methods we compare against in this paper. The horizontal axis represents time, gray rectangles mark when each technique requires environment interactions. For BCO, the white and gray circles denote the inverse model and policy learning steps, respectively. For BCO, jIprejis the number of post-demonstration environment interactions performed before each model- and policy-improvement step and for IRL methods, jIIRLjrepresents the total number of interactions. 3 Problem Formulation We consider agents acting within the broad framework of Markov decision processes (MDPs). We denote a MDP us- ing the 5-tuple M=fS;A;T;r; g, whereSis the agent’s state space, Ais its action space, Tsia si+1=P(si+1jsi;a)is a function denoting the probability of the agent transitioning from statesitosi+1after taking action a,r:SA!R is a function specifying the immediate reward that the agent receives for taking a specific action in a given state, and is a discount factor. In this framework, agent behavior can be specified by a policy, :S!A, which specifies the ac- tion (or distribution over actions) that the agent should use when in a particular state. We denote the set of state transi- tions experienced by an agent during a particular execution of a policybyT=f(si;si+1)g. In the context of these transitions, we will be interested in the inverse dynamics model, Msisi+1a =P(ajsi;si+1), which is the probability of having taken action agiven that the agent transitioned from state sitosi+1. Moreover, we specifically seek task-independent models. We assume that some of the state features are specifically related to the task and others specifically to the agent, i.e., a given state scan be partitioned into an agent-specific state, sa, and a task-specific state,st, which are members of sets SaandSt, respectively (i.e.,S=SaSt)[Konidaris, 2006; Gupta et al., 2017 ]. Using this partitioning, we define the agent-specific inverse dynamics model to be a functionM:SaSa!p(A)that maps a pair of agent-specific state transitions, (sa i;sa i+1)2 Ta , to a distribution of agent actions that is likely to have given rise to that transition. Imitation learning is typically defined in the context of a MDP without an explicitly-defined reward function, i.e., Mnr. The learning problem is for an agent to determine an imitation policy, :S!Athat the agent may use in order to behave like the expert, using a provided set of expert demon- strationsf1;2;:::gin which each is a demonstrated state- action trajectoryf(s0;a0);(s1;a1);:::;(sN;aN)g. There- fore, in this setting, the agent must have access to the demon- strator’s actions. If the imitator is unable to observe the action sequences used by the demonstrator, the resulting imitation learning problem has recently been referred to as imitationfrom observation [Liuet al., 2017 ]. In this setting, one seeks to find an imitation policy from a set of state-only demon- strationsD=f1;2;:::gin which each is a state-only trajectoryfs0;s1;:::;sNg. The specific problem that we are interested in is imitation from observation under a constrained number of environment interactions. By environment interactions we mean time steps for which we require our agent to gather new data by execut- ing an action in its environment and observing the state out- come. We are concerned here in particular with the cost of the learning process, in terms of the number of environment interactions, both before and after the expert demonstrations are provided. Pre- and post-demonstration environment in- teractions are represented by IpreandIpost, respectively, to denote sets of interactions (si;ai;si+1)that must be executed by a learner before and after a demonstration becomes avail- able. In this context, we are concerned here with the fol- lowing specific goal: given a set of state-only demonstration trajectories, D, find a good imitation policy using a minimal number of post-demonstration environment interactions, i.e., jIpostj. In pursuit of this goal, we propose a new algorithm for imitation learning that can operate both in the absence of demonstrator action information and while requiring no or very few post-demonstration environment interactions. Our framework consists of two components, each of which con- siders a separate part of this problem. The first of these com- ponents considers the problem of learning an agent-specific inverse dynamics model, and the second one considers the problem of learning an imitation policy from a set of demon- stration trajectories. 4 Behavioral Cloning from Observation We now describe our imitation learning algorithm, BCO, which combines inverse dynamics model learning with learn- ing an imitation policy. We are motivated by the fact that humans have access to a large amount of prior experience about themselves, and so we aim to also provide an au- tonomous agent with this same prior knowledge. To do so, before any demonstration information is observed, we allow the agent to learn its own agent-specific inverse dynamicsProceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18) 4952model. Then, given state-only demonstration information, we use this learned model to infer the expert’s missing action in- formation. Once these actions have been inferred, the agent performs imitation learning via a modified version of behav- ioral cloning (Figure 1). The pseudo-code of the algorithm is given in Algorithm 1. 4.1 Inverse Dynamics Model Learning In order to infer missing action information, we first allow the agent to acquire prior experience in the form of an agent- specific inverse dynamics model. In order to do so, we let the agent perform an exploration policy, . In this work, we let be a random policy (Algorithm 1, Line 2). While execut- ing this policy, the agent performs some number of interac- tions with the environment, i.e., Ipre. Because we seek an agent-specific inverse dynamics model as described in Sec- tion 3, we extract the agent-specific part of the states in Ipre and store them as Ta e=f(sa i;sa i+1)g, and their associated actions,Ae=faig(Algorithm 1, Lines 5-8). Given this in- formation, the problem of learning an agent-specific inverse dynamics model is that of finding the parameter for which Mbest describes the observed transitions. We formulate this problem as one of maximum-likelihood estimation, i.e., we seekas =arg maxjIprej i=0p(aijsa i;sa i+1); (1) wherepis the conditional distribution over actions induced byMgiven a specific state transition. Any number of su- pervised learning techniques, denoted as “modelLearning” in Algorithm 1, may be used to solve (1). Some details regarding particular choices made in this pa- per: For domains with a continuous action space, we assume a Gaussian distribution over each action dimension and our model estimates the individual means and standard devia- tions. We use a neural network for M, where the network receives a state transition as input and outputs the mean for each action dimension. The standard deviation is also learned for each dimension, but it is computed independently of the state transitions. In order to train this network (i.e., to find  in (1)), we use the Adam variant [Kingma and Ba, 2014 ]of stochastic gradient decent. Intuitively, the gradient for each sample is computed by finding a change in that would in- crease the probability of aiwith respect to the distribution specified byM(si;si+1). When the action space is discrete, we again use a neural network for M, where the network computes the probability of taking each action via a softmax function. 4.2 Behavioral Cloning Our overarching problem is that of finding a good imita- tion policy from a set of state-only demonstration trajecto- ries,Ddemo =f1;2;:::gwhere each is a trajectory fs0;s1;:::;sNg. Note that, although the inverse dynamics model is learned using a set of agent-generated data in Sec- tion 4.1, the data used there is not utilized in this step. In or- der to use the learned agent-specific inverse dynamics model, we first extract the agent-specific part of the demonstrated state sequences and then form the set of demonstrated agent- specific state transitions Ta demo (Algorithm 1, Line 10). Next,Algorithm 1 BCO( ) 1:Initialize the model Mas random approximator 2:Setto be a random policy 3:SetI=jIprej 4:while policy improvement do 5: fortime-step t=1 to Ido 6: Generate samples (sa t;sa t+1)andatusing 7: Append samplesTa  (sa t;sa t+1),A at 8: end for 9: ImproveMby modelLearning(Ta ,A) 10: Generate set of agent-specific state transitions Ta demo from the demonstrated state trajectories Ddemo 11: UseMwithTa demo to approximate ~Ademo 12: Improveby behavioralCloning(S demo;~Ademo ) 13: SetI=jIprej 14:end while for each transition (sa i;sa i+1)2Ta demo , the algorithm com- putes the model-predicted distribution over demonstrator ac- tions,M(sa i;sa i+1)and uses the maximum-likelihood ac- tion as the inferred action, ~ai, which is placed in a set ~Ademo (Algorithm 1, Line 11). Using these inferred actions, we then build the set of complete state-action pairs f(si;~ai)g. With this new set of state-action pairs, we may now seek the imitation policy . We cast this problem as one of behav- ioral cloning, i.e., given a set of state-action tuples f(si;~ai)g, the problem of learning an imitation policy becomes that of finding the parameter for whichbest matches this set of provided state-action pairs (Algorithm 1, Line 12). We find this parameter using maximum-likelihood estimation, i.e., we seekas =arg maxN i=0(~aijsi): (2) Some details regarding particular choices made in this pa- per: For continuous action spaces, we assume our policy to be Gaussian over each action dimension, and, for discrete ac- tions spaces we use a softmax function to represent the prob- ability of selecting each value. We let be a neural network that receives as input a state and outputs either the Gaussian distribution parameters or the action probabilities for contin- uous or discrete action spaces, respectively. We then solve forin (2) using Adam SGD, where the intuitive view of the gradient is that it seeks to find changes in that increase the probability of each inferred demonstrator action, ~ai, in the imitation policy’s distribution (
jsi). 4.3 Model Improvement The techniques described above form the building blocks of BCO. If one is willing to tolerate post-demonstration en- vironment interaction, a modified version of our algorithm can further improve both the learned model and the resulting imitation policy. This modified algorithm proceeds as fol- lows. After the behavioral cloning step, the agent executes the imitation policy in the environment for a short period of time. Then, the newly-observed state-action sequences are used to update the model, and, accordingly, the imitation policy itself. The above procedure is repeated until there isProceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18) 4953(a) CartPole (b) MountainCar (c) Reacher (d) Ant Figure 3: Representative screenshots of the domains considered in this paper. no more improvement in the imitation policy. We call this modified version of our algorithm BCO( ), whereis a user-specified parameter that is used to control the number of post-demonstration environment interactions at each iter- ation,M, according to M=jIprej. The total number of post-demonstration interactions required by BCO( ) can be calculated asjIprej=TM =TjIprej, whereTis the total number of model-improvement iterations required by BCO( ). Using a nonzero , the model is able to leverage post-demonstration environment interaction in order to more accurately estimate the actions taken by the demonstrator, and therefore improve its learned imitation policy. If one has a fixed budget for post-demonstration interactions, one could consider terminating the model-improvement iterations early, i.e., specify both andT. 5 Implementation and Experimental Results We evaluated BCO( ) in several domains available in Ope- nAI Gym [Brockman et al., 2016 ]. Continuous tasks are sim- ulated by MuJoCo [Todorov et al., 2012 ]. These domains have different levels of difficulty, as measured by the com- plexity of the dynamics and the size and continuity of the state and action spaces. Ordered from easy to hard, the domains we considered are: CartPole, MountainCar, Reacher, and Ant-v1. Each of these domains has predefined state features, actions and reward functions in the simulator. Any RL algo- rithm could be used with this reward function to generate an expert policy. Since trust region policy optimization (TRPO) [Schulman et al., 2015 ]has shown promising performance in simulation [Liuet al., 2017 ], we generated our demonstra- tions using agents trained with this method. We evaluated our algorithm in two senses. First, with re- spect to the number of environment interactions required to attain a certain performance. In a real-world environment, in- teractions can be expensive which makes it a very important criterion. The second way in which we evaluate our algo- rithm is with respect to data efficiency, i.e., the imitator’s task performance as a function of the amount of available demon- stration data. In general, demonstration data is scarce, and so making the best use of it is very important. We compared BCO( ) to the following methods: 1.Behavioral Cloning (BC): This method applies super- vised learning over state-action pairs provided by the demonstrator. 2.Feature Expectation Matching (FEM) [Hoet al.,2016]:A modified version of the approach presented by Abbeel and Ng [2004]. It uses trust region policy opti- mization with a linear cost function in order to train train neural network policies. 3.General Adversarial Imitation Learning (GAIL) [Ho and Ermon, 2016 ]:A state-of-the-art in IRL. It uses a specific class of cost functions which allows for the use of generative adversarial networks in order to do appren- ticeship learning. Note in particular that our method is the only method that does not have access to the demonstrator’s actions. How- ever, as our results will show, BCO( ) can still achieve com- parable performance to these other techniques, and do so while requiring far fewer environment interactions. 5.1 Training Details and Results Because both BC and BCO( ) rely on supervised learning methods, we use only 70% of the available data for training and use the rest for validation. We stop training when the er- ror on the 30% validation data starts to increase. For the other methods, all available data was used in the training process. We will now discuss the architecture, details for each domain. CartPole: The goal is to keep the pole vertically upward as long as possible (Figure 3a). This domain has a dis- crete action space. In this domain, we considered linear models over the pre-defined state features for both the inverse dynamics model and the imitation policy and we only usedM= 1000 interactions to learn the dynamics. MountainCar: The goal is to have the car reach the target point (Figure 3b). This domain has a discrete action space. In this domain, the data set for learning the inverse dynamics model is acquired by letting the agent to explore its action space for M= 2000 time steps. For both the imitation policy and inverse dynam- ics model, we used neural networks with two hidden lay- ers, 8 nodes each, and leaky rectified linear activation functions (LReLU). Reacher: This domain has a continuous action space. The goal is to have the fingertip of the arm reach the tar- get point whose position changes in every episode (Fig- ure 3c). Therefore, in this domain, we can partition the state-space to agent-specific features (i.e., those only re- lated to the arm) and task-specific features (i.e., those related to the position of the target). A neural networkProceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18) 49540.0 0.5 1.0 1.5 1e60.00.20.40.60.81.0Performance1e3 interactionsCartPole* Random ExpertGAIL FEMBCO(0)0.0 0.5 1.0 1.5 1e60.00.20.40.60.81.0 2e3 interactionsMountainCar 0.0 0.2 0.4 0.6 0.8 1.0 1e7−2.0−1.5−1.0−0.50.00.51.0 5e3 interactionsReacher** 0.0 0.5 1.0 1.5 2.0 2.5 1e7−1.5−1.0−0.50.00.51.0 5e5 interactionsAnt Number of interactionsFigure 4: Performance of each technique with respect to the number of post-demonstration interactions. For each domain, ten demonstrated trajectories were considered. BCO(0) is depicted as a horizontal line since all environment interactions happen before the demonstration is provided. Performance values are scaled such that performance of a random policy is zero and the performance of the expert is one. Note that GAIL and FEM have access to demonstration action information whereas BCO does not. *The BCO line is not visible for the CartPole domain because BCO has the same performance as the expert. **FEM is not shown for the Reacher domain because its performance is much worse than the other techniques. . 1 4 7 100.00.20.40.60.81.0PerformanceCartPole 1 4 7 100.00.20.40.60.81.0MountainCar 1 5 10 150.00.20.40.60.81.0Reacher* 5 10 15 20 25−1.5−1.0−0.50.00.51.0Ant Number of demonstrated trajectoriesRandom ExpertFEM GAILBCO(0)** BC Figure 5: Performance of imitation agents with respect to the number of available demonstration trajectories. Rectangular bars and error bars represent the mean return and the standard error, respectively, as measured over 5000 trajectories. Returns have been scaled such that the performance of a random policy and the demonstrating agent are zero and one, respectively. *Note that FEM is not shown for the Reacher domain because its performance is much worse than the others. **Note that BC, GAIL, and FEM all have access to demonstration action information whereas BCO(0) does not. architecture with two hidden layers of 100 LReLU nodes are used with M= 5000 agent-specific state transition- action pairs in order to learn the dynamics and then this model is used to learn a policy which also has two layers but with 32 LReLU nodes. Ant: The goal to have the ant to run as fast as pos- sible(Figure 3d). This domain has a continuous action space. This is the most complex domain considered in this work. The state and action space are 111 and 8 dimensional, respectively. The number of interactions needed to learn the dynamics was M= 5e5and the ar- chitectures for inverse dynamics learning and the policy are similar to those we used in Reacher. 5.2 Discussion Each experiment was executed twenty times, and all results presented here are the average values over these twenty runs. We selected twenty trials because we empirically observed very small standard error bars in our results. This is likely areflection of the relatively low variance in the expert demon- strations. In our first experiment, we compare the number of en- vironment interactions needed for BCO(0) with the num- ber required by other methods (Figure 4.) We can clearly see how imitation performance improves as the agent is able to interact more with the environment. In the case of BCO(0), the interactions with the environment happen be- fore the policy-learning process starts, and so we represent its performance with a horizontal line. The height of the line indicates the performance, and we display the number of pre- demonstration environment interactions it required next to it. The random and expert policies also do not benefit from post- demonstration environment interaction, and so they are also shown using horizontal lines. From these plots, we can see that it takes at least 40 times more interactions required by GAIL or FEM to gain the same performance as BCO(0). Now, we aim to compare the performance of our algo- rithm BCO( ) with the other algorithms. To do so, we useProceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18) 49551 4 7 100.00.20.40.60.8PerformanceMountainCar 1 5 10 150.00.20.40.60.81.0Reacher 5 10 15 20 250.00.20.40.60.81.0Ant Number of demonstrated trajectoriesα= 0 α= 2E − 3α= 4E − 3 α= 1E − 2BCFigure 6: The performance of BC and several BCO( ) techniques (varying ) with respect to the number of demonstrated trajectories provided. Rectangular bars and error bars represent the mean return and the standard error, respectively, as measured over 5000 trajectories. By increasing , more post-demonstration environment interactions are allowed to occur, which increases the performance of the imitation policy. Note that BC has access to demonstration action information whereas BCO does not. Also note that the number of trajectories required for learning a fairly good policy is very small. Each demonstrated trajectory has 5, 50, and 50 transitions for each domain from left to right, respectively. Note that we did not demonstrate the results for CartPole because the results were equally perfect regardless of the value of . MountainCar (pre-demo = 2E 3) Reacher (pre-demo = 5E 3) Ant (pre-demo = 5E5) dn2E34E31E2dn2E34E31E2dn2E34E31E2 1 6825 23475 28950 1 210052 358736 912368 5 602500 1270000 3362500 4 8387 12000 31200 5 270500 486578 1837500 10 940000 2075000 5000000 7 6300 23100 122200 10 221421 569736 1055921 15 1387500 2855000 7325000 10 45462 61450 88600 15 509289 852210 1859210 20 1925000 4055000 9687500 Table 1: This is an extension to Figure 6 which provides the number of post-demonstration interactions for each case. First vertical column for each domain (d) shows the number of demonstrated trajectories. As an example, for the MountainCar domain with 1demonstrated trajectory and= 2E3, the average number of post-demonstration interactions is 6825. We can also see at the top of the table that the number of pre-demonstration interactions is 2E3so the overall number of interactions would become 8825 . It can be seen that almost always by increasingor the number of demonstrated trajectories, the number of post-demonstration interactions increases. Also the overall number of interactions (combined pre- and post-demonstration interactions) in all the cases is far less than the overall number of interactions required by the other methods (FEM and GAIL). each algorithm to train the agents, and then calculate the fi- nal performance by computing the average return over 5000 episodes. For comparison purposes, we scale these perfor- mance values such that the performance of the expert and a random policy are 1:0and0:0, respectively. This compari- son is shown in Figures 5, and 6 where we have plotted the performance of each algorithm with respect to the number of available demonstrated trajectories. Figure 5 shows the com- parison between the performance of BCO(0) with all the other methods, and Figure 6 compares the performance of BCO( ) across different values of . In Figure 5, we can see that performance of our method is comparable with other meth- ods even though our method is the only one without access to the actions. In the case of Reacher, the transferability of the learned inverse model is highlighted by the high performance of BCO. In the case of the Reacher and Ant domains, we can see that FEM performs poorly compared to others, per- haps because the rewards are not simple enough to be approx- imated by linear functions. In the CartPole domain, each of the methods performs as well as the expert and so all the linesare over each other. In the case of MountainCar, our perfor- mance is worse than other methods. Conversely, for Reacher, ours is more sample efficient than GAIL, i.e., with smaller number of demonstrations we get much better results. In the case of Ant, our method performs almost as good as GAIL. In Figure 6, we can see that BCO’s performance improves with largersince the extra environment interactions allow it to make better estimates of the demonstrator’s actions. 6 Conclusions In this paper, we have presented BCO, an algorithm for per- forming imitation learning that requires neither access to demonstrator actions nor post-demonstration environment in- teraction. Our experimental results show that the result- ing imitation policies perform favorably compared to those generated by existing imitation learning approaches that do require access to demonstrator actions. Moreover, BCO requires fewer post-demonstration environment interactions than these other techniques, meaning that a reasonable imita- tion policy can be executed with less delay.Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18) 4956Acknowledgments Peter Stone serves on the Board of Directors of Cogitai, Inc. The terms of this arrangement have been reviewed and ap- proved by the University of Texas at Austin in accordance with its policy on objectivity in research. References [Argall et al., 2009 ]Brenna D Argall, Sonia Chernova, Manuela Veloso, and Brett Browning. A survey of robot learning from demonstration. Robotics and autonomous systems, 57(5):469–483, 2009. [Bain and Sommut, 1999 ]Michael Bain and Claude Som- mut. A framework for behavioural claning. Machine In- telligence 15, 15:103, 1999. [Bojarski et al., 2016 ]Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp, Pra- soon Goyal, Lawrence D Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, et al. End to end learning for self- driving cars. arXiv preprint arXiv:1604.07316, 2016. [Brockman et al., 2016 ]Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. OpenAI Gym. arXiv preprint arXiv:1606.01540, 2016. [Chebotar et al., 2017 ]Yevgen Chebotar, Karol Hausman, Marvin Zhang, Gaurav Sukhatme, Stefan Schaal, and Sergey Levine. Combining model-based and model- free updates for trajectory-centric reinforcement learning. arXiv preprint arXiv:1703.03078, 2017. [Daftry et al., 2016 ]Shreyansh Daftry, J Andrew Bagnell, and Martial Hebert. Learning transferable policies for monocular reactive mav control. In International Sym- posium on Experimental Robotics, pages 3–11. Springer, 2016. [Finn et al., 2016 ]Chelsea Finn, Sergey Levine, and Pieter Abbeel. Guided cost learning: Deep inverse optimal con- trol via policy optimization. In International Conference on Machine Learning, pages 49–58, 2016. [Giusti et al., 2016 ]Alessandro Giusti, J ´erˆome Guzzi, Dan C Cires ¸an, Fang-Lin He, Juan P Rodr ´ıguez, Flavio Fontana, Matthias Faessler, Christian Forster, J ¨urgen Schmidhuber, Gianni Di Caro, et al. A machine learning approach to visual perception of forest trails for mobile robots. IEEE Robotics and Automation Letters, 1(2):661–667, 2016. [Gupta et al., 2017 ]Abhishek Gupta, Coline Devin, YuXuan Liu, Pieter Abbeel, and Sergey Levine. Learning invariant feature spaces to transfer skills with reinforcement learn- ing. arXiv preprint arXiv:1703.02949, 2017. [Hanna and Stone, 2017 ]Josiah P Hanna and Peter Stone. Grounded action transformation for robot learning in sim- ulation. In AAAI, pages 3834–3840, 2017. [Ho and Ermon, 2016 ]Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning. In Advances in Neural Information Processing Systems, pages 4565– 4573, 2016.[Hoet al., 2016 ]Jonathan Ho, Jayesh Gupta, and Stefano Ermon. Model-free imitation learning with policy opti- mization. In International Conference on Machine Learn- ing, pages 2760–2769, 2016. [Kingma and Ba, 2014 ]Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. [Konidaris, 2006 ]George D Konidaris. A framework for transfer in reinforcement learning. In ICML-06 Workshop on Structural Knowledge Transfer for Machine Learning, 2006. [Liuet al., 2017 ]YuXuan Liu, Abhishek Gupta, Pieter Abbeel, and Sergey Levine. Imitation from observation: Learning to imitate behaviors from raw video via context translation. arXiv preprint arXiv:1707.03374, 2017. [Niekum et al., 2015 ]Scott Niekum, Sarah Osentoski, George Konidaris, Sachin Chitta, Bhaskara Marthi, and Andrew G Barto. Learning grounded finite-state representations from unstructured demonstrations. The In- ternational Journal of Robotics Research, 34(2):131–157, 2015. [Ross and Bagnell, 2010 ]St´ephane Ross and Drew Bagnell. Efficient reductions for imitation learning. In Proceedings of the thirteenth international conference on artificial in- telligence and statistics, pages 661–668, 2010. [Ross et al., 2011 ]St´ephane Ross, Geoffrey J Gordon, and Drew Bagnell. A reduction of imitation learning and struc- tured prediction to no-regret online learning. In Interna- tional Conference on Artificial Intelligence and Statistics, pages 627–635, 2011. [Schaal, 1997 ]Stefan Schaal. Learning from demonstration. InAdvances in neural information processing systems, pages 1040–1046, 1997. [Schulman et al., 2015 ]John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. Trust region policy optimization. In Proceedings of the 32nd In- ternational Conference on Machine Learning (ICML-15), pages 1889–1897, 2015. [Sutton and Barto, 1998 ]Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction, vol- ume 1. MIT press Cambridge, 1998. [Taylor et al., 2008 ]Matthew E Taylor, Nicholas K Jong, and Peter Stone. Transferring instances for model- based reinforcement learning. In Joint European Confer- ence on Machine Learning and Knowledge Discovery in Databases, pages 488–505. Springer, 2008. [Todorov et al., 2012 ]Emanuel Todorov, Tom Erez, and Yu- val Tassa. Mujoco: A physics engine for model-based control. In Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on, pages 5026–5033. IEEE, 2012.Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18) 4957

- Paper ID: 07263bee15626561168a1919102ceb2e790f5499 Title: Retrospective surprise: A computational component for active inference Abs... - Paper ID: 07263bee15626561168a1919102ceb2e790f5499 Title: Retrospective surprise: A computational component for active inference Abstract: In the free energy principle (FEP), proposed by Friston, it is supposed that agents seek to minimize the “surprise”–the negative log (marginal) likelihood of observations (i.e., sensory stimuli)–given the agents' current belief. This is achieved by minimizing the free energy, which provides an upper bound on the surprise. The FEP has been applied to action selection in a framework called “active inference,” where agents are supposed to select an action so that they minimize the “expected free energy” (EFE). While the FEP and active inference have attracted the attention of researchers in a wide range of fields such as psychology and psychiatry, as well as neuroscience, it is not clear which psychological construct EFE is related to. To facilitate the discussion and interpretation of psychological processes underlying active inference, w... - Paper ID: 07263bee15626561168a1919102ceb2e790f5499 Title: Retrospective surprise: A computational component for active inference Abstract: In the free energy principle (FEP), proposed by Friston, it is supposed that agents seek to minimize the “surprise”–the negative log (marginal) likelihood of observations (i.e., sensory stimuli)–given the agents' current belief. This is achieved by minimizing the free energy, which provides an upper bound on the surprise. The FEP has been applied to action selection in a framework called “active inference,” where agents are supposed to select an action so that they minimize the “expected free energy” (EFE). While the FEP and active inference have attracted the attention of researchers in a wide range of fields such as psychology and psychiatry, as well as neuroscience, it is not clear which psychological construct EFE is related to. To facilitate the discussion and interpretation of psychological processes underlying active inference, we introduce a computational component termed the “retrospective (or residual) surprise,” which is the surprise of an observation after updating the belief given the observation itself. We show that the predicted retrospective surprise (PRS) provides a lower bound on EFE: EFE is always larger than PRS. We illustrate the properties of EFE and PRS using examples of inference for a binary hidden cause given a binary observation. Essentially, EFE and PRS show similar behavior; however, in certain situations, they provide different predictions regarding action selection. This study also provides insights into the mechanism of active inference based on EFE. Citation Count: 0 - Paper ID: 8e91147687ba84903569ad9a3993d73ee2f3e3d9 Title: Immunity-Based Intrusion Detection System: A General Framework Abstract: This paper focuses on investigating immunological principles in designing a multi-agent system for intrusion/anomaly detection and response in networked computers. In this approach, the immunity-based agents roam around the machines (nodes or routers), and monitor the situation in the network (i.e. look for changes such as malfunctions, faults, abnormalities, misuse, deviations, intrusions, etc.). These agents can mutually recognize each other's activities and can take appropriate actions according to the underlying security policies. Specifically, their activities are coordinated in a hierarchical fashion while sensing, communicating and generating responses. Such an agent can learn and adapt to its environment dynamically and can detect both known and unknown intrusions. This research is the part of an effort to develop a multi-agent detection system that can simultaneously monitor networked computer's activities at different levels (such as user level, system level, process level and packet level) in order to determine intrusions and anomalies. The proposed intrusion detection system is designed to be flexible, extendible, and adaptable that can perform real-time monitoring in accordance with the needs and preferences of network administrators. This paper provides the conceptual view and a general framework of the proposed system. 1. Inspiration from the nature: Every organism in nature is constantly threatened by other organisms, and each species has evolved elaborate set of protective measures called, collectively, the immune system. The natural immune system is an adaptive learning system that is highly distributive in nature. It employs multi-level defense mechanisms to make rapid, highly specific and often very protective responses against wide variety of pathogenic microorganisms. The immune system is a subject of great research interest because of its powerful information processing capabilities [5,6]. Specifically, its' mechanisms to extract unique signatures from antigens and ability to recognize and classify dangerous antigenic peptides are very important. It also uses memory to remember signature patterns that have been seen previously, and use combinatorics to construct antibody for efficient detection. It is observed that the overall behavior of the system is an emergent property of several local interactions. Moreover, the immune response can be either local or systemic, depending on the route and property of the antigenic challenge [19]. The immune system is consists of different populations of immune cells (mainly B or T cells) which circulate at various primary and secondary lymphoid organs of the body. They are carefully controlled to ensure that appropriate populations of B and T cells (naive, effector, and memory) are recruited into different location [19]. This differential migration of lymphocyte subpopulations at different locations (organs) of the body is called trafficking or homing. The lymph nodes and organs provide specialized local environment (called germinal center) during pathogenic attack in any part of the body. This dynamic mechanism support to create a large number of antigen-specific lymphocytes (as effector and memory cells) for stronger defense through the process of the clonal expansion and differentiation. Interestingly, memory cells exhibit selective homing to the type of tissue in which they first encountered an antigen. Presumably this ensures that a particular memory cell will return to the location where it is most likely to re-encounter a subsequent antigenic challenge. The mechanisms of immune responses are self-regulatory in nature. There is no central organ that controls the functions of the immune system. The regulation of the clonal expansion and proliferation of B cells are closely regulated (with a co-stimulation) in order to prevent uncontrolled immune response. This second signal helps to ensure tolerance and judge between dangerous and harmless invaders. So the purpose of this accompanying signal in identifying a non-self is to minimize false alarm and to generate decisive response in case of a real danger[19]. 2. Existing works in Intrusion Detection: The study of security in computer networks is a rapidly growing area of interest because of the proliferation of networks (LANs, WANs etc.), greater deployment of shared computer databases (packages) and the increasing reliance of companies, institutions and individuals on such data. Though there are many levels of access protection to computing and network resources, yet the intruders are finding ways to entry into many sites and systems, and causing major damages. So the task of providing and maintaining proper security in a network system becomes a challenging issue. Intrusion/Anomaly detection is an important part of computer security. It provides an additional layer of defense against computer misuse (abuse) after physical, authentication and access control. There exist different methods for intrusion detection [7,23,25,29] and the early models include IDES (later versions NIDES and MIDAS), W & S, AudES, NADIR, DIDS, etc. These approaches monitor audit trails generated by systems and user applications and perform various statistical analyses in order to derive regularities in behavior pattern. These works based on the hypothesis that an intruder's behavior will be noticeably different from that of a legitimate user, and security violations can be detected by monitoring these audit trails. Most of these methods, however, used to monitor a single host [13,14], though NADIR and DIDS can collect and aggregate audit data from a number of hosts to detect intrusions. However, in all cases, there is no real analysis of patterns of network activities and they only perform centralized analysis. Recent works include GrIDS[27] which used hierarchical graphs to detect attacks on networked systems. Other approaches used autonomous agent architectures [1,2,26] for distributed intrusion detection. 3. Computer Immune Systems: The security in the field of computing may be considered as analogous to the immunity in natural systems. In computing, threats and dangers (of compromising privacy, integrity, and availability) may arise because of malfunction of components or intrusive activities (both internal and external). The idea of using immunological principles in computer security [9-11,15,16,18] started since 1994. Stephanie Forrest and her group at the University of New Mexico have been working on a research project with a long-term goal to build an artificial immune system for computers [911,15,16]. This immunity-based system has much more sophisticated notions of identity and protection than those afforded by current operating systems, and it is suppose to provide a general-purpose protection system to augment current computer security systems. The security of computer systems depends on such activities as detecting unauthorized use of computer facilities, maintaining the integrity of data files, and preventing the spread of computer viruses. The problem of protecting computer systems from harmful viruses is viewed as an instance of the more general problem of distinguishing self (legitimate users, uncorrupted data, etc.) from dangerous other (unauthorized users, viruses, and other malicious agents). This method (called the negative-selection algorithm) is intended to be complementary to the more traditional cryptographic and deterministic approaches to computer security. As an initial step, the negativeselection algorithm has been used as a file-authentication method on the problem of computer virus detection [9]. Citation Count: 287 - Paper ID: 193c4795d7e14c714c3b048b76419a2dc756c31d Title: Learning Faults Detection by AIS Techniques in CSCL Environments Abstract: By the increase of e-learning platforms, huge data sets are made from different kinds of the collected traces. These traces differ from one learner to another according to their characteristics (learning styles, preferences, performed actions, etc.). Learners' traces are very heterogeneous and voluminous, so their treatments and exploitations are difficult, that make hard the tutors' tasks. This paper introduces one of the bio-inspired computing techniques to improve the learning quality. In fact, Artificial Immune System (AIS) is a technique which was adapted for designing an assistant system that detects the wrong scenarios made by learners. Furthermore, this assistant system assists the learners in their activities. The main aim is to present the basic concepts of a new approach that aims at providing learners with relevant traces to improve their learning in order to minimize the tutor's tasks. A novel algorithm is proposed to design the assistant system based on the two mechanisms of the AIS techniques (negative and clonal selection). The proposed algorithm was applied on a collaborative learning system called LETline 2.0 (http://www.labstic.com/letline/). An experiment was conducted in an Algerian University. The obtained results from this experiment were good and very efficient. The proposed approach enhances the cognitive and behavioral profiles of learners. In fact, the results show that the cognitive profiles of most students were improved. Also, it minimizes the tutor's tasks. Citation Count: 4

You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you s... You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you select really corresponds to the excerpt: there will be details mentioned in the excerpt that should appear in the paper. If you read an abstract and it seems like it could be the paper we’re looking for, read the paper to make sure. Also: sometimes you’ll read a paper that cites the paper we’re looking for. In such cases, please go to the references in order to find the full name of the paper we’re looking for, and search for it, and then select it. Specifically, although [CITATION] first demonstrated that there exists a modality gap between text and image embeddings generated from VLMs, the geometry of this modality gap permits cross-modality transferability. This phenomenon allows text to serve as a proxy to corresponding images and vice versa. You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you select really corresponds to the excerpt: there will be details mentioned in the excerpt that should appear in the paper. If you read an abstract and it seems like it could be the paper we’re looking for, read the paper to make sure. Also: sometimes you’ll read a paper that cites the paper we’re looking for. In such cases, please go to the references in order to find the full name of the paper we’re looking for, and search for it, and then select it. Specifically, although [CITATION] first demonstrated that there exists a modality gap between text and image embeddings generated from VLMs, the geometry of this modality gap permits cross-modality transferability. This phenomenon allows text to serve as a proxy to corresponding images and vice versa.

- Paper ID: b6cf4579b59b51d7df416e096ad86c1e6a48b458 Title: Adversarial Prompt Tuning for Vision-Language Models Abstract: With the ra... - Paper ID: b6cf4579b59b51d7df416e096ad86c1e6a48b458 Title: Adversarial Prompt Tuning for Vision-Language Models Abstract: With the rapid advancement of multimodal learning, pre-trained Vision-Language Models (VLMs) such as CLIP have demonstrated remarkable capacities in bridging the gap between visual and language modalities. However, these models remain vulnerable to adversarial attacks, particularly in the image modality, presenting considerable security risks. This paper introduces Adversarial Prompt Tuning (AdvPT), a novel technique to enhance the adversarial robustness of image encoders in VLMs. AdvPT innovatively leverages learnable text prompts and aligns them with adversarial image embeddings, to address the vulnerabilities inherent in VLMs without the need for extensive parameter training or modification of the model architecture. We demonstrate that AdvPT improves resistance against white-box and black-box adversarial attacks and exhibits a synergistic effect when comb... - Paper ID: b6cf4579b59b51d7df416e096ad86c1e6a48b458 Title: Adversarial Prompt Tuning for Vision-Language Models Abstract: With the rapid advancement of multimodal learning, pre-trained Vision-Language Models (VLMs) such as CLIP have demonstrated remarkable capacities in bridging the gap between visual and language modalities. However, these models remain vulnerable to adversarial attacks, particularly in the image modality, presenting considerable security risks. This paper introduces Adversarial Prompt Tuning (AdvPT), a novel technique to enhance the adversarial robustness of image encoders in VLMs. AdvPT innovatively leverages learnable text prompts and aligns them with adversarial image embeddings, to address the vulnerabilities inherent in VLMs without the need for extensive parameter training or modification of the model architecture. We demonstrate that AdvPT improves resistance against white-box and black-box adversarial attacks and exhibits a synergistic effect when combined with existing image-processing-based defense techniques, further boosting defensive capabilities. Comprehensive experimental analyses provide insights into adversarial prompt tuning, a novel paradigm devoted to improving resistance to adversarial images through textual input modifications, paving the way for future robust multimodal learning research. These findings open up new possibilities for enhancing the security of VLMs. Our code is available at https://github.com/jiamingzhang94/Adversarial-Prompt-Tuning. Citation Count: 5

- Paper ID: e937251952897fa8839515742faf5d42ccc93ba9 Title: Diagram Visual Grounding: Learning to See with Gestalt-Perceptual Attention ... - Paper ID: e937251952897fa8839515742faf5d42ccc93ba9 Title: Diagram Visual Grounding: Learning to See with Gestalt-Perceptual Attention Abstract: Diagram visual grounding aims to capture the correlation between language expression and local objects in the diagram, and plays an important role in the applications like textbook question answering and cross-modal retrieval. Most diagrams consist of several colors and simple geometries. This results in sparse low-level visual features, which further aggravates the gap between low-level visual and high-level semantic features of diagrams. The phenomenon brings challenges to the diagram visual grounding. To solve the above issues, we propose a gestalt-perceptual attention model to align the diagram objects and language expressions. For low-level visual features, inspired by the gestalt that simulates human visual system, we build a gestalt-perception graph network to make up the features learned by the traditional backbone network. For ... - Paper ID: e937251952897fa8839515742faf5d42ccc93ba9 Title: Diagram Visual Grounding: Learning to See with Gestalt-Perceptual Attention Abstract: Diagram visual grounding aims to capture the correlation between language expression and local objects in the diagram, and plays an important role in the applications like textbook question answering and cross-modal retrieval. Most diagrams consist of several colors and simple geometries. This results in sparse low-level visual features, which further aggravates the gap between low-level visual and high-level semantic features of diagrams. The phenomenon brings challenges to the diagram visual grounding. To solve the above issues, we propose a gestalt-perceptual attention model to align the diagram objects and language expressions. For low-level visual features, inspired by the gestalt that simulates human visual system, we build a gestalt-perception graph network to make up the features learned by the traditional backbone network. For high-level semantic features, we design a multi-modal context attention mechanism to facilitate the interaction between diagrams and language expressions, so as to enhance the semantics of diagrams. Finally, guided by diagram features and linguistic embedding, the target query is gradually decoded to generate the coordinates of the referred object. By conducting comprehensive experiments on diagrams and natural images, we demonstrate that the proposed model achieves superior performance over the competitors. Our code will be released at https://github.com/AIProCode/GPA. Citation Count: 0 - Paper ID: e738c4b4060e459c13646dee70b753809b82391b Title: A Modular Vision Language Navigation and Manipulation Framework for Long Horizon Compositional Tasks in Indoor Environment Abstract: In this paper we propose a new framework—MoViLan (Modular Vision and Language) for execution of visually grounded natural language instructions for day to day indoor household tasks. While several data-driven, end-to-end learning frameworks have been proposed for targeted navigation tasks based on the vision and language modalities, performance on recent benchmark data sets revealed the gap in developing comprehensive techniques for long horizon, compositional tasks (involving manipulation and navigation) with diverse object categories, realistic instructions and visual scenarios with non reversible state changes. We propose a modular approach to deal with the combined navigation and object interaction problem without the need for strictly aligned vision and language training data (e.g., in the form of expert demonstrated trajectories). Such an approach is a significant departure from the traditional end-to-end techniques in this space and allows for a more tractable training process with separate vision and language data sets. Specifically, we propose a novel geometry-aware mapping technique for cluttered indoor environments, and a language understanding model generalized for household instruction following. We demonstrate a significant increase in success rates for long horizon, compositional tasks over recent works on the recently released benchmark data set -ALFRED. Citation Count: 7

- Paper ID: f5c165b6317896a65151050201c737536fa17c31 Title: mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-c... - Paper ID: f5c165b6317896a65151050201c737536fa17c31 Title: mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections Abstract: Large-scale pre-trained foundation models have been an emerging paradigm for building artificial intelligence (AI) systems, which can be quickly adapted to a wide range of downstream tasks. This paper presents mPLUG, a new vision-language foundation model for both cross-modal understanding and generation. Most existing pre-trained models suffer from inefficiency and linguistic signal overwhelmed by long visual sequences in cross-modal alignment. To address both problems, mPLUG introduces an effective and efficient vision-language architecture with novel cross-modal skip-connections.mPLUG is pre-trained end-to-end on large-scale image-text pairs with both discriminative and generative objectives. It achieves state-of-the-art results on a wide range of vision-language downstream tasks, including image captioning, image-text re... - Paper ID: f5c165b6317896a65151050201c737536fa17c31 Title: mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections Abstract: Large-scale pre-trained foundation models have been an emerging paradigm for building artificial intelligence (AI) systems, which can be quickly adapted to a wide range of downstream tasks. This paper presents mPLUG, a new vision-language foundation model for both cross-modal understanding and generation. Most existing pre-trained models suffer from inefficiency and linguistic signal overwhelmed by long visual sequences in cross-modal alignment. To address both problems, mPLUG introduces an effective and efficient vision-language architecture with novel cross-modal skip-connections.mPLUG is pre-trained end-to-end on large-scale image-text pairs with both discriminative and generative objectives. It achieves state-of-the-art results on a wide range of vision-language downstream tasks, including image captioning, image-text retrieval, visual grounding and visual question answering. mPLUG also demonstrates strong zero-shot transferability on vision-language and video-language tasks. The code and pre-trained models are available at https://github.com/alibaba/AliceMind Citation Count: 122 - Paper ID: 1e1b08b4583c24b1be1e413cfec29a572454b190 Title: Leveraging Visual Knowledge in Language Tasks: An Empirical Study on Intermediate Pre-training for Cross-Modal Knowledge Transfer Abstract: Pre-trained language models are still far from human performance in tasks that need understanding of properties (e.g. appearance, measurable quantity) and affordances of everyday objects in the real world since the text lacks such information due to reporting bias.In this work, we study whether integrating visual knowledge into a language model can fill the gap.We investigate two types of knowledge transfer: (1) text knowledge transfer using image captions that may contain enriched visual knowledge and (2) cross-modal knowledge transfer using both images and captions with vision-language training objectives.On 5 downstream tasks that may need visual knowledge to solve the problem, we perform extensive empirical comparisons over the presented objectives.Our experiments show that visual knowledge transfer can improve performance in both low-resource and fully supervised settings. Citation Count: 8 - Paper ID: 979c3ba88b3e3604d5a9ccfcc6a8a608704b868b Title: RelCLIP: Adapting Language-Image Pretraining for Visual Relationship Detection via Relational Contrastive Learning Abstract: Conventional visual relationship detection models only use the numeric ids of relation labels for training, but ignore the semantic correlation between the labels, which leads to severe training biases and harms the generalization ability of representations. In this paper, we introduce compact language information of relation labels for regularizing the representation learning of visual relations. Specifically, we propose a simple yet effective visual Relationship prediction framework that transfers natural language knowledge learned from Contrastive Language-Image Pre-training (CLIP) models to enhance the relationship prediction, termed RelCLIP. Benefiting from the powerful visual-semantic alignment ability of CLIP at image level, we introduce a novel Relational Contrastive Learning (RCL) approach which explores relation-level visual-semantic alignment via learning to match cross-modal relational embeddings. By collaboratively learning the semantic coherence and discrepancy from relation triplets, the model can generate more discriminative and robust representations. Experimental results on the Visual Genome dataset show that RelCLIP achieves significant improvements over strong baselines under full (provide accurate labels) and distant supervision (provide noise labels), demonstrating its powerful generalization ability in learning relationship representations. Code will be available at https://gitee.com/mindspore/models/tree/master/research/cv/RelCLIP. Citation Count: 2 - Paper ID: 5e00596fa946670d894b1bdaeff5a98e3867ef13 Title: SimVLM: Simple Visual Language Model Pretraining with Weak Supervision Abstract: With recent progress in joint modeling of visual and textual representations, Vision-Language Pretraining (VLP) has achieved impressive performance on many multimodal downstream tasks. However, the requirement for expensive annotations including clean image captions and regional labels limits the scalability of existing approaches, and complicates the pretraining procedure with the introduction of multiple dataset-specific objectives. In this work, we relax these constraints and present a minimalist pretraining framework, named Simple Visual Language Model (SimVLM). Unlike prior work, SimVLM reduces the training complexity by exploiting large-scale weak supervision, and is trained end-to-end with a single prefix language modeling objective. Without utilizing extra data or task-specific customization, the resulting model significantly outperforms previous pretraining methods and achieves new state-of-the-art results on a wide range of discriminative and generative vision-language benchmarks, including VQA (+3.74% vqa-score), NLVR2 (+1.17% accuracy), SNLI-VE (+1.37% accuracy) and image captioning tasks (+10.1% average CIDEr score). Furthermore, we demonstrate that SimVLM acquires strong generalization and transfer ability, enabling zero-shot behavior including open-ended visual question answering and cross-modality transfer. Citation Count: 607 - Paper ID: 141a5033d9994242b18bb3b217e79582f1ee9306 Title: Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Abstract: Pre-trained representations are becoming crucial for many NLP and perception tasks. While representation learning in NLP has transitioned to training on raw text without human annotations, visual and vision-language representations still rely heavily on curated training datasets that are expensive or require expert knowledge. For vision applications, representations are mostly learned using datasets with explicit class labels such as ImageNet or OpenImages. For vision-language, popular datasets like Conceptual Captions, MSCOCO, or CLIP all involve a non-trivial data collection (and cleaning) process. This costly curation process limits the size of datasets and hence hinders the scaling of trained models. In this paper, we leverage a noisy dataset of over one billion image alt-text pairs, obtained without expensive filtering or post-processing steps in the Conceptual Captions dataset. A simple dual-encoder architecture learns to align visual and language representations of the image and text pairs using a contrastive loss. We show that the scale of our corpus can make up for its noise and leads to state-of-the-art representations even with such a simple learning scheme. Our visual representation achieves strong performance when transferred to classification tasks such as ImageNet and VTAB. The aligned visual and language representations enables zero-shot image classification and also set new state-of-the-art results on Flickr30K and MSCOCO image-text retrieval benchmarks, even when compared with more sophisticated cross-attention models. The representations also enable cross-modality search with complex text and text + image queries. Citation Count: 2435 - Paper ID: ad8b899d359240fca7c623cb0c6b8b03309e48fa Title: CL2CM: Improving Cross-Lingual Cross-Modal Retrieval via Cross-Lingual Knowledge Transfer Abstract: Cross-lingual cross-modal retrieval has garnered increasing attention recently, which aims to achieve the alignment between vision and target language (V-T) without using any annotated V-T data pairs. Current methods employ machine translation (MT) to construct pseudo-parallel data pairs, which are then used to learn a multi-lingual and multi-modal embedding space that aligns visual and target-language representations. However, the large heterogeneous gap between vision and text, along with the noise present in target language translations, poses significant challenges in effectively aligning their representations. To address these challenges, we propose a general framework, Cross-Lingual to Cross-Modal (CL2CM), which improves the alignment between vision and target language using cross-lingual transfer. This approach allows us to fully leverage the merits of multi-lingual pre-trained models (e.g., mBERT) and the benefits of the same modality structure, i.e., smaller gap, to provide reliable and comprehensive semantic correspondence (knowledge) for the cross-modal network. We evaluate our proposed approach on two multilingual image-text datasets, Multi30K and MSCOCO, and one video-text dataset, VATEX. The results clearly demonstrate the effectiveness of our proposed method and its high potential for large-scale retrieval. Citation Count: 0 - Paper ID: 0f98609d8d5caad25b90b51e92025ba2bbde1b6a Title: FM-OV3D: Foundation Model-based Cross-modal Knowledge Blending for Open-Vocabulary 3D Detection Abstract: The superior performances of pre-trained foundation models in various visual tasks underscore their potential to enhance the 2D models' open-vocabulary ability. Existing methods explore analogous applications in the 3D space. However, most of them only center around knowledge extraction from singular foundation models, which limits the open-vocabulary ability of 3D models. We hypothesize that leveraging complementary pre-trained knowledge from various foundation models can improve knowledge transfer from 2D pre-trained visual language models to the 3D space. In this work, we propose FM-OV3D, a method of Foundation Model-based Cross-modal Knowledge Blending for Open-Vocabulary 3D Detection, which improves the open-vocabulary localization and recognition abilities of 3D model by blending knowledge from multiple pre-trained foundation models, achieving true open-vocabulary without facing constraints from original 3D datasets. Specifically, to learn the open-vocabulary 3D localization ability, we adopt the open-vocabulary localization knowledge of the Grounded-Segment-Anything model. For open-vocabulary 3D recognition ability, We leverage the knowledge of generative foundation models, including GPT-3 and Stable Diffusion models, and cross-modal discriminative models like CLIP. The experimental results on two popular benchmarks for open-vocabulary 3D object detection show that our model efficiently learns knowledge from multiple foundation models to enhance the open-vocabulary ability of the 3D model and successfully achieves state-of-the-art performance in open-vocabulary 3D object detection tasks. Code is released at https://github.com/dmzhang0425/FM-OV3D.git. Citation Count: 1 - Paper ID: 2ab8d3a547d6a806d75332bae0915d4f37a41d1e Title: Test-Time Adaptation for Visual Document Understanding Abstract: For visual document understanding (VDU), self-supervised pretraining has been shown to successfully generate transferable representations, yet, effective adaptation of such representations to distribution shifts at test-time remains to be an unexplored area. We propose DocTTA, a novel test-time adaptation method for documents, that does source-free domain adaptation using unlabeled target document data. DocTTA leverages cross-modality self-supervised learning via masked visual language modeling, as well as pseudo labeling to adapt models learned on a \textit{source} domain to an unlabeled \textit{target} domain at test time. We introduce new benchmarks using existing public datasets for various VDU tasks, including entity recognition, key-value extraction, and document visual question answering. DocTTA shows significant improvements on these compared to the source model performance, up to 1.89\% in (F1 score), 3.43\% (F1 score), and 17.68\% (ANLS score), respectively. Our benchmark datasets are available at \url{https://saynaebrahimi.github.io/DocTTA.html}. Citation Count: 5 - Paper ID: 17ca48ad1b944c897863f04ba9ffa72674dce1ce Title: Parallel multi-head attention and term-weighted question embedding for medical visual question answering Abstract: None Citation Count: 3 - Paper ID: 4d747a0458e98d0351842a606c0bab7867b070e6 Title: Language-assisted Vision Model Debugger: A Sample-Free Approach to Finding and Fixing Bugs Abstract: Vision models with high overall accuracy often exhibit systematic errors in specific scenarios, posing potential serious safety concerns. Diagnosing bugs of vision models is gaining increased attention, however traditional diagnostic approaches require annotation efforts (eg rich metadata accompanying each samples of CelebA). To address this issue,We propose a language-assisted diagnostic method that uses texts instead of images to diagnose bugs in vision models based on multi-modal models (eg CLIP). Our approach connects the embedding space of CLIP with the buggy vision model to be diagnosed; meanwhile, utilizing a shared classifier and the cross-modal transferability of embedding space from CLIP, the text-branch of CLIP become a proxy model to find bugs in the buggy model. The proxy model can classify texts paired with images. During the diagnosis, a Large Language Model (LLM) is employed to obtain task-relevant corpora, and this corpora is used to extract keywords. Descriptions constructed with templates containing these keywords serve as input text to probe errors in the proxy model. Finally, we validate the ability to diagnose existing visual models using language on the Waterbirds and CelebA datasets, we can identify bugs comprehensible to human experts, uncovering not only known bugs but also previously unknown ones. Citation Count: 0

- Paper ID: 637ac919087ea5822313189bf929f45893e08291 Title: CMA-CLIP: Cross-Modality Attention Clip for Text-Image Classification Abst... - Paper ID: 637ac919087ea5822313189bf929f45893e08291 Title: CMA-CLIP: Cross-Modality Attention Clip for Text-Image Classification Abstract: Multi-modal learning with both text and images benefits multiple applications, such as attribute extraction for e-commerce products. In this paper, we propose Cross-Modality Attention Contrastive Language-Image Pre-training (CMA-CLIP), a new multi-modal architecture to jointly learn the fine-grained inter-modality relationship. It fuses CLIP with a sequence-wise attention module and a modality-wise attention module. The network uses CLIP to bridge the inter-modality gap at the global level, and uses the sequence-wise attention module to capture the fine-grained alignment between text and images. Besides, it leverages a modality-wise attention module to learn the relevance of each modality to downstream tasks, making the network robust against irrelevant modalities. CMA-CLIP outperforms the state-of-the-art method on Fashion-Gen by 5.5% in acc... - Paper ID: 637ac919087ea5822313189bf929f45893e08291 Title: CMA-CLIP: Cross-Modality Attention Clip for Text-Image Classification Abstract: Multi-modal learning with both text and images benefits multiple applications, such as attribute extraction for e-commerce products. In this paper, we propose Cross-Modality Attention Contrastive Language-Image Pre-training (CMA-CLIP), a new multi-modal architecture to jointly learn the fine-grained inter-modality relationship. It fuses CLIP with a sequence-wise attention module and a modality-wise attention module. The network uses CLIP to bridge the inter-modality gap at the global level, and uses the sequence-wise attention module to capture the fine-grained alignment between text and images. Besides, it leverages a modality-wise attention module to learn the relevance of each modality to downstream tasks, making the network robust against irrelevant modalities. CMA-CLIP outperforms the state-of-the-art method on Fashion-Gen by 5.5% in accuracy, achieves competitive performance on Food101 and performance on par with the state-of-the-art method on MM-IMDb. We also demonstrate CMA-CLIP’s robustness against irrelevant modalities on an Amazon dataset for the task of product attribute extraction. Citation Count: 14 - Paper ID: f4d0f97e29bfe31bc744bd204c4fed248536dace Title: Bridging the Gap: Multi-Level Cross-Modality Joint Alignment for Visible-Infrared Person Re-Identification Abstract: Visible-Infrared person Re-IDentification (VI-ReID) is a challenging cross-modality image retrieval task that aims to match pedestrians' images across visible and infrared cameras. To solve the modality gap, existing mainstream methods adopt a learning paradigm converting the image retrieval task into an image classification task with cross-entropy loss and auxiliary metric learning losses. These losses follow the strategy of adjusting the distribution of extracted embeddings to reduce the intra-class distance and increase the inter-class distance. However, such objectives do not precisely correspond to the final test setting of the retrieval task, resulting in a new gap at the optimization level. By rethinking these keys of VI-ReID, we propose a simple and effective method, the Multi-level Cross-modality Joint Alignment (MCJA), bridging both modality and objective-level gap. For the former, we design the Modality Alignment Augmentation, which consists of three novel strategies, the weighted grayscale, cross-channel cutmix, and spectrum jitter augmentation, effectively reducing modality discrepancy in the image space. For the latter, we introduce a new Cross-Modality Retrieval loss. It is the first work to constrain from the perspective of the ranking list, aligning with the goal of the testing stage. Moreover, based on the global feature only, our method exhibits good performance and can serve as a strong baseline method for the VI-ReID community. Citation Count: 0 - Paper ID: 0d47f107827c9f38142c0365c67e1a156ae421ab Title: PEIT: Bridging the Modality Gap with Pre-trained Models for End-to-End Image Translation Abstract: Image translation is a task that translates an image containing text in the source language to the target language. One major challenge with image translation is the modality gap between visual text inputs and textual inputs/outputs of machine translation (MT). In this paper, we propose PEIT, an end-to-end image translation framework that bridges the modality gap with pre-trained models. It is composed of four essential components: a visual encoder, a shared encoder-decoder backbone network, a vision-text representation aligner equipped with the shared encoder and a cross-modal regularizer stacked over the shared decoder. Both the aligner and regularizer aim at reducing the modality gap. To train PEIT, we employ a two-stage pre-training strategy with an auxiliary MT task: (1) pre-training the MT model on the MT training data to initialize the shared encoder-decoder backbone network; and (2) pre-training PEIT with the aligner and regularizer on a synthesized dataset with rendered images containing text from the MT training data. In order to facilitate the evaluation of PEIT and promote research on image translation, we create a large-scale image translation corpus ECOIT containing 480K image-translation pairs via crowd-sourcing and manual post-editing from real-world images in the e-commerce domain. Experiments on the curated ECOIT benchmark dataset demonstrate that PEIT substantially outperforms both cascaded image translation systems (OCR+MT) and previous strong end-to-end image translation model, with fewer parameters and faster decoding speed. Citation Count: 1 - Paper ID: ea10d0a86d0989d97088964d9dfb3f3dbc34daa2 Title: Reducing Domain Gap in Frequency and Spatial domain for Cross-modality Domain Adaptation on Medical Image Segmentation Abstract: Unsupervised domain adaptation (UDA) aims to learn a model trained on source domain and performs well on unlabeled target domain. In medical image segmentation field, most existing UDA methods depend on adversarial learning to address the domain gap between different image modalities, which is ineffective due to its complicated training process. In this paper, we propose a simple yet effective UDA method based on frequency and spatial domain transfer under multi-teacher distillation framework. In the frequency domain, we first introduce non-subsampled contourlet transform for identifying domain-invariant and domain-variant frequency components (DIFs and DVFs), and then keep the DIFs unchanged while replacing the DVFs of the source domain images with that of the target domain images to narrow the domain gap. In the spatial domain, we propose a batch momentum update-based histogram matching strategy to reduce the domain-variant image style bias. Experiments on two commonly used cross-modality medical image segmentation datasets show that our proposed method achieves superior performance compared to state-of-the-art methods. Citation Count: 4 - Paper ID: 70afa614ee4ac8080b950b714a1d9c93404d6fe7 Title: Mind the Gap: Alleviating Local Imbalance for Unsupervised Cross-Modality Medical Image Segmentation Abstract: Unsupervised cross-modality medical image adaptation aims to alleviate the severe domain gap between different imaging modalities without using the target domain label. A key in this campaign relies upon aligning the distributions of source and target domain. One common attempt is to enforce the global alignment between two domains, which, however, ignores the fatal local-imbalance domain gap problem, i.e., some local features with larger domain gap are harder to transfer. Recently, some methods conduct alignment focusing on local regions to improve the efficiency of model learning. While this operation may cause a deficiency of critical information from contexts. To tackle this limitation, we propose a novel strategy to alleviate the domain gap imbalance considering the characteristics of medical images, namely Global-Local Union Alignment. Specifically, a feature-disentanglement style-transfer module first synthesizes the target-like source images to reduce the global domain gap. Then, a local feature mask is integrated to reduce the ‘inter-gap’ for local features by prioritizing those discriminative features with larger domain gap. This combination of global and local alignment can precisely localize the crucial regions in segmentation target while preserving the overall semantic consistency. We conduct a series of experiments with two cross-modality adaptation tasks, i,e. cardiac substructure and abdominal multi-organ segmentation. Experimental results indicate that our method achieves state-of-the-art performance in both tasks. Citation Count: 2 - Paper ID: fa2ae8f0b9d15bb55094785bc49f59c338d1edc6 Title: Learning cross-modality features for image caption generation Abstract: None Citation Count: 3 - Paper ID: 11d98606b8b0f5ed686bc16e6079b030411abd3b Title: Multi‐level cross‐modality learning framework for text‐based person re‐identification Abstract: The target of text‐based person re‐identification (Re‐ID) is to retrieve the corresponding image of a person through the given text information. However, due to the homogeneous variety and modality heterogeneity, it is challenging to simultaneously learn both global‐level and local‐level cross‐modal features and align them in the same embedding space without additional networks. To address these problems, an effective multi‐level cross‐modality learning (MCL) framework for language and vision person Re‐ID is proposed. More specifically, a multi‐branch feature extraction (MFE) module is designed to comprehensively map both global and partial semantic information for the visual and textual embedding at the same time, capturing the intra‐class semantic relationships in multi‐granularities. Besides, a cross‐modal alignment (CA) module is devised to match the multi‐grained representations and reduce the inter‐class gap from global‐level to partial‐level. Extensive experiments conducted on the CUHK‐PEDES and ICFG‐PEDES datasets suggest that this method outperforms the state‐of‐the‐art models. Citation Count: 1 - Paper ID: b32c4a0a5e401cb565d12d5b30485b21aeb38570 Title: Cross-modality Data Augmentation for End-to-End Sign Language Translation Abstract: End-to-end sign language translation (SLT) aims to convert sign language videos into spoken language texts directly without intermediate representations. It has been a challenging task due to the modality gap between sign videos and texts and the data scarcity of labeled data. To tackle these challenges, we propose a novel Cross-modality Data Augmentation (XmDA) framework to transfer the powerful gloss-to-text translation capabilities to end-to-end sign language translation (i.e. video-to-text) by exploiting pseudo gloss-text pairs from the sign gloss translation model. Specifically, XmDA consists of two key components, namely, cross-modality mix-up and cross-modality knowledge distillation. The former explicitly encourages the alignment between sign video features and gloss embeddings to bridge the modality gap. The latter utilizes the generation knowledge from gloss-to-text teacher models to guide the spoken language text generation. Experimental results on two widely used SLT datasets, i.e., PHOENIX-2014T and CSL-Daily, demonstrate that the proposed XmDA framework significantly and consistently outperforms the baseline models. Extensive analyses confirm our claim that XmDA enhances spoken language text generation by reducing the representation distance between videos and texts, as well as improving the processing of low-frequency words and long sentences. Citation Count: 6 - Paper ID: 7e39112e704d7c088b592fd5b0fde331594843ae Title: U NDERSTANDING THE MODALITY GAP IN CLIP Abstract: This work examines the phenomenon of the modality gap observed in CLIP-based multimodal learning methods. The modality gap in this context refers to the separation of image and text embeddings in the joint latent space. Some previous research has attributed the gap to cone effect of neural network initialization and suggested closing may not be necessary. However, this study argues that the modality gap is associated with local minima in the CLIP loss function. Through a series of proof-of-concept experiments, we illustrate these local minima and the difficulty of avoiding them in practice. Overall, this work hopes to provide better insight into the root cause of the modality gap. Citation Count: 1 - Paper ID: 5e4ab2acf6420c6333aa75d424ed6e5b137549f9 Title: Cross-Modality Transfer Learning for Image-Text Information Management Abstract: In the past decades, information from all kinds of data has been on a rapid increase. With state-of-the-art performance, machine learning algorithms have been beneficial for information management. However, insufficient supervised training data is still an adversity in many real-world applications. Therefore, transfer learning (TF) was proposed to address this issue. This article studies a not well investigated but important TL problem termed cross-modality transfer learning (CMTL). This topic is closely related to distant domain transfer learning (DDTL) and negative transfer. In general, conventional TL disciplines assume that the source domain and the target domain are in the same modality. DDTL aims to make efficient transfers even when the domains or the tasks are entirely different. As an extension of DDTL, CMTL aims to make efficient transfers between two different data modalities, such as from image to text. As the main focus of this study, we aim to improve the performance of image classification by transferring knowledge from text data. Previously, a few CMTL algorithms were proposed to deal with image classification problems. However, most existing algorithms are very task specific, and they are unstable on convergence. There are four main contributions in this study. First, we propose a novel heterogeneous CMTL algorithm, which requires only a tiny set of unlabeled target data and labeled source data with associate text tags. Second, we introduce a latent semantic information extraction method to connect the information learned from the image data and the text data. Third, the proposed method can effectively handle the information transfer across different modalities (text-image). Fourth, we examined our algorithm on a public dataset, Office-31. It has achieved up to 5% higher classification accuracy than “non-transfer” algorithms and up to 9% higher than existing CMTL algorithms. Citation Count: 15

This paper does not have an open access PDF. This paper does not have an open access PDF. This paper does not have an open access PDF.

- Paper ID: 6d73f9af8a1470b0d52dfcbeca1f2625b31a0c1d Title: Bridging Modality Gap for Visual Grounding with Effecitve Cross-modal Distill... - Paper ID: 6d73f9af8a1470b0d52dfcbeca1f2625b31a0c1d Title: Bridging Modality Gap for Visual Grounding with Effecitve Cross-modal Distillation Abstract: Visual grounding aims to align visual information of specific regions of images with corresponding natural language expressions. Current visual grounding methods leverage pre-trained visual and language backbones separately to obtain visual features and linguistic features. Although these two types of features are then fused via delicately designed networks, the heterogeneity of the features makes them inapplicable for multi-modal reasoning. This problem arises from the domain gap between the single-modal pre-training backbone used in current visual grounding methods, which can hardly be overcome by the traditional end-to-end training method. To alleviate this, our work proposes an Empowering pre-trained model for Visual Grounding (EpmVG) framework, which distills a multimodal pre-trained model to guide the visual grounding task.... - Paper ID: 6d73f9af8a1470b0d52dfcbeca1f2625b31a0c1d Title: Bridging Modality Gap for Visual Grounding with Effecitve Cross-modal Distillation Abstract: Visual grounding aims to align visual information of specific regions of images with corresponding natural language expressions. Current visual grounding methods leverage pre-trained visual and language backbones separately to obtain visual features and linguistic features. Although these two types of features are then fused via delicately designed networks, the heterogeneity of the features makes them inapplicable for multi-modal reasoning. This problem arises from the domain gap between the single-modal pre-training backbone used in current visual grounding methods, which can hardly be overcome by the traditional end-to-end training method. To alleviate this, our work proposes an Empowering pre-trained model for Visual Grounding (EpmVG) framework, which distills a multimodal pre-trained model to guide the visual grounding task. EpmVG is based on a novel cross-modal distillation mechanism, which can effectively introduce the consistency information of images and texts in the pre-trained model, to reduce the domain gap existing in the backbone networks, thereby improving the performance of the model in the visual grounding task. Extensive experiments are carried out on five conventionally used datasets, and results demonstrate that our method achieves better performance than state-of-the-art methods. Citation Count: 0 - Paper ID: 2313afae52d98e569da2dedbf14daf9efc74e7cf Title: CogVLM: Visual Expert for Pretrained Language Models Abstract: We introduce CogVLM, a powerful open-source visual language foundation model. Different from the popular shallow alignment method which maps image features into the input space of language model, CogVLM bridges the gap between the frozen pretrained language model and image encoder by a trainable visual expert module in the attention and FFN layers. As a result, CogVLM enables deep fusion of vision language features without sacrificing any performance on NLP tasks. CogVLM-17B achieves state-of-the-art performance on 10 classic cross-modal benchmarks, including NoCaps, Flicker30k captioning, RefCOCO, RefCOCO+, RefCOCOg, Visual7W, GQA, ScienceQA, VizWiz VQA and TDIUC, and ranks the 2nd on VQAv2, OKVQA, TextVQA, COCO captioning, etc., surpassing or matching PaLI-X 55B. Codes and checkpoints are available at https://github.com/THUDM/CogVLM. Citation Count: 127 - Paper ID: 0d47f107827c9f38142c0365c67e1a156ae421ab Title: PEIT: Bridging the Modality Gap with Pre-trained Models for End-to-End Image Translation Abstract: Image translation is a task that translates an image containing text in the source language to the target language. One major challenge with image translation is the modality gap between visual text inputs and textual inputs/outputs of machine translation (MT). In this paper, we propose PEIT, an end-to-end image translation framework that bridges the modality gap with pre-trained models. It is composed of four essential components: a visual encoder, a shared encoder-decoder backbone network, a vision-text representation aligner equipped with the shared encoder and a cross-modal regularizer stacked over the shared decoder. Both the aligner and regularizer aim at reducing the modality gap. To train PEIT, we employ a two-stage pre-training strategy with an auxiliary MT task: (1) pre-training the MT model on the MT training data to initialize the shared encoder-decoder backbone network; and (2) pre-training PEIT with the aligner and regularizer on a synthesized dataset with rendered images containing text from the MT training data. In order to facilitate the evaluation of PEIT and promote research on image translation, we create a large-scale image translation corpus ECOIT containing 480K image-translation pairs via crowd-sourcing and manual post-editing from real-world images in the e-commerce domain. Experiments on the curated ECOIT benchmark dataset demonstrate that PEIT substantially outperforms both cascaded image translation systems (OCR+MT) and previous strong end-to-end image translation model, with fewer parameters and faster decoding speed. Citation Count: 1 - Paper ID: 5d87a2c55eb1583c747a4d05dc898391e1ddf77b Title: MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition Abstract: Multi-media communications facilitate global interaction among people. However, despite researchers exploring cross-lingual translation techniques such as machine translation and audio speech translation to overcome language barriers, there is still a shortage of cross-lingual studies on visual speech. This lack of research is mainly due to the absence of datasets containing visual speech and translated text pairs. In this paper, we present AVMuST-TED, the first dataset for Audio-Visual Multilingual Speech Translation, derived from TED talks. Nonetheless, visual speech is not as distinguishable as audio speech, making it difficult to develop a mapping from source speech phonemes to the target language text. To address this issue, we propose MixSpeech, a cross-modality self-learning framework that utilizes audio speech to regularize the training of visual speech tasks. To further minimize the cross-modality gap and its impact on knowledge transfer, we suggest adopting mixed speech, which is created by interpolating audio and visual streams, along with a curriculum learning strategy to adjust the mixing ratio as needed. MixSpeech enhances speech translation in noisy environments, improving BLEU scores for four languages on AVMuST-TED by +1.4 to +4.2. Moreover, it achieves state-of-the-art performance in lip reading on CMLR (11.1%), LRS2 (25.5%), and LRS3 (28.0%). Citation Count: 14 - Paper ID: 132679d2fff8f8aed41bf8388a39b7f0aa30d4fd Title: Set-level Guidance Attack: Boosting Adversarial Transferability of Vision-Language Pre-training Models Abstract: Vision-language pre-training (VLP) models have shown vulnerability to adversarial examples in multimodal tasks. Furthermore, malicious adversaries can be deliberately transferred to attack other black-box models. However, existing work has mainly focused on investigating white-box attacks. In this paper, we present the first study to investigate the adversarial transferability of recent VLP models. We observe that existing methods exhibit much lower transferability, compared to the strong attack performance in white-box settings. The transferability degradation is partly caused by the under-utilization of cross-modal interactions. Particularly, unlike unimodal learning, VLP models rely heavily on cross-modal interactions and the multimodal alignments are many-to-many, e.g., an image can be described in various natural languages. To this end, we propose a highly transferable Set-level Guidance Attack (SGA) that thoroughly leverages modality interactions and incorporates alignment-preserving augmentation with cross-modal guidance. Experimental results demonstrate that SGA could generate adversarial examples that can strongly transfer across different VLP models on multiple downstream vision-language tasks. On image-text retrieval, SGA significantly enhances the attack success rate for transfer attacks from ALBEF to TCL by a large margin (at least 9.78% and up to 30.21%), compared to the state-of-the-art. Our code is available at https://github.com/Zoky-2020/SGA. Citation Count: 17 - Paper ID: b32c4a0a5e401cb565d12d5b30485b21aeb38570 Title: Cross-modality Data Augmentation for End-to-End Sign Language Translation Abstract: End-to-end sign language translation (SLT) aims to convert sign language videos into spoken language texts directly without intermediate representations. It has been a challenging task due to the modality gap between sign videos and texts and the data scarcity of labeled data. To tackle these challenges, we propose a novel Cross-modality Data Augmentation (XmDA) framework to transfer the powerful gloss-to-text translation capabilities to end-to-end sign language translation (i.e. video-to-text) by exploiting pseudo gloss-text pairs from the sign gloss translation model. Specifically, XmDA consists of two key components, namely, cross-modality mix-up and cross-modality knowledge distillation. The former explicitly encourages the alignment between sign video features and gloss embeddings to bridge the modality gap. The latter utilizes the generation knowledge from gloss-to-text teacher models to guide the spoken language text generation. Experimental results on two widely used SLT datasets, i.e., PHOENIX-2014T and CSL-Daily, demonstrate that the proposed XmDA framework significantly and consistently outperforms the baseline models. Extensive analyses confirm our claim that XmDA enhances spoken language text generation by reducing the representation distance between videos and texts, as well as improving the processing of low-frequency words and long sentences. Citation Count: 6 - Paper ID: bb876cb814fe0e14ead87ca0cd651f3c7c1153b1 Title: Bidirectional Cross-Modal Knowledge Exploration for Video Recognition with Pre-trained Vision-Language Models Abstract: Vision-language models (VLMs) pre-trained on large- scale image-text pairs have demonstrated impressive transferability on various visual tasks. Transferring knowledge from such powerful VLMs is a promising direction for building effective video recognition models. However, current exploration in this field is still limited. We believe that the greatest value of pre-trained VLMs lies in building a bridge between visual and textual domains. In this paper, we propose a novel framework called BIKE, which utilizes the cross-modal bridge to explore bidirectional knowledge: i) We introduce the Video Attribute Association mechanism, which leverages the Video-to-Text knowledge to generate textual auxiliary attributes for complementing video recognition. ii) We also present a Temporal Concept Spotting mechanism that uses the Text-to-Video expertise to capture temporal saliency in a parameter-free manner, leading to enhanced video representation. Extensive studies on six popular video datasets, including Kinetics-400 & 600, UCF-101, HMDB-51, ActivityNet and Charades, show that our method achieves state-of-the-art performance in various recognition scenarios, such as general, zero-shot, and few-shot video recognition. Our best model achieves a state-of-the-art accuracy of 88.6% on the challenging Kinetics-400 using the released CLIP model. The code is available at https://github.com/whwu95/BIKE. Citation Count: 28 - Paper ID: bb1bc9970a52566134280d2b01e0920c80d3d53e Title: Cross-modality Multiple Relations Learning for Knowledge-based Visual Question Answering Abstract: Knowledge-based visual question answering not only needs to answer the questions based on images but also incorporates external knowledge to study reasoning in the joint space of vision and language. To bridge the gap between visual content and semantic cues, it is important to capture the question-related and semantics-rich vision-language connections. Most existing solutions model simple intra-modality relation or represent cross-modality relation using a single vector, which makes it difficult to effectively model complex connections between visual features and question features. Thus, we propose a cross-modality multiple relations learning model, aiming to better enrich cross-modality representations and construct advanced multi-modality knowledge triplets. First, we design a simple yet effective method to generate multiple relations that represent the rich cross-modality relations. The various cross-modality relations link the textual question to the related visual objects. These multi-modality triplets efficiently align the visual objects and corresponding textual answers. Second, to encourage multiple relations to better align with different semantic relations, we further formulate a novel global-local loss. The global loss enables the visual objects and corresponding textual answers close to each other through cross-modality relations in the vision-language space, and the local loss better preserves semantic diversity among multiple relations. Experimental results on the Outside Knowledge VQA and Knowledge-Routed Visual Question Reasoning datasets demonstrate that our model outperforms the state-of-the-art methods. Citation Count: 1 - Paper ID: fb24091f419a0daaf213653858a69ab05d0186aa Title: Bridge the Gap between Language models and Tabular Understanding Abstract: Table pretrain-then-finetune paradigm has been proposed and employed at a rapid pace after the success of pre-training in the natural language domain. Despite the promising findings in tabular pre-trained language models (TPLMs), there is an input gap between pre-training and fine-tuning phases. For instance, TPLMs jointly pre-trained with table and text input could be effective for tasks also with table-text joint input like table question answering, but it may fail for tasks with only tables or text as input such as table retrieval. To this end, we propose UTP, an approach that dynamically supports three types of multi-modal inputs: table-text, table, and text. Specifically, UTP is pre-trained with two strategies: (1) We first utilize a universal mask language modeling objective on each kind of input, enforcing the model to adapt various inputs. (2) We then present Cross-Modal Contrastive Regularization (CMCR), which utilizes contrastive learning to encourage the consistency between table-text cross-modality representations via unsupervised instance-wise training signals during pre-training. By these means, the resulting model not only bridges the input gap between pre-training and fine-tuning but also advances in the alignment of table and text. Extensive results show UTP achieves superior results on uni-modal input tasks (e.g., table retrieval) and cross-modal input tasks (e.g., table question answering). Citation Count: 4 - Paper ID: fdf8f703710a6237ee29ab794086844bf245ab0e Title: VT-CLIP: Enhancing Vision-Language Models with Visual-guided Texts Abstract: Contrastive Language-Image Pre-training (CLIP) has drawn increasing attention recently for its transferable visual representation learning. However, due to the semantic gap within datasets, CLIP's pre-trained image-text alignment becomes sub-optimal on downstream tasks, which severely harms its transferring performance. To better adapt the cross-modality embedding space, we propose to enhance CLIP via Visual-guided Texts, named VT-CLIP. Specifically, we guide textual features of different categories to adaptively explore informative regions on the image and aggregate visual features by attention mechanisms. In this way, the texts become visual-guided, namely, more semantically correlated with downstream images, which greatly benefits the category-wise matching process. In few-shot settings, we evaluate our VT-CLIP on 11 well-known classification datasets to demonstrate its effectiveness. Citation Count: 31

You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you s... You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you select really corresponds to the excerpt: there will be details mentioned in the excerpt that should appear in the paper. If you read an abstract and it seems like it could be the paper we’re looking for, read the paper to make sure. Also: sometimes you’ll read a paper that cites the paper we’re looking for. In such cases, please go to the references in order to find the full name of the paper we’re looking for, and search for it, and then select it. [CITATION] show: No computable learner can non-uniformly learn the class $\mathcal{H}_{comp}$, the class of all computable functions from $\mathbb{N}$ to {0, 1} You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you select really corresponds to the excerpt: there will be details mentioned in the excerpt that should appear in the paper. If you read an abstract and it seems like it could be the paper we’re looking for, read the paper to make sure. Also: sometimes you’ll read a paper that cites the paper we’re looking for. In such cases, please go to the references in order to find the full name of the paper we’re looking for, and search for it, and then select it. [CITATION] show: No computable learner can non-uniformly learn the class $\mathcal{H}_{comp}$, the class of all computable functions from $\mathbb{N}$ to {0, 1}

You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you s... You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you select really corresponds to the excerpt: there will be details mentioned in the excerpt that should appear in the paper. If you read an abstract and it seems like it could be the paper we’re looking for, read the paper to make sure. Also: sometimes you’ll read a paper that cites the paper we’re looking for. In such cases, please go to the references in order to find the full name of the paper we’re looking for, and search for it, and then select it. Significant progress was made by [CITATION], who gave a characterization of proper strong computable PAC learning in terms of the computability of a Empirical Risk Minimizer (ERM) and who constructed a class of finite VC dimension which is not computable PAC learnable, even in the improper sense. You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you select really corresponds to the excerpt: there will be details mentioned in the excerpt that should appear in the paper. If you read an abstract and it seems like it could be the paper we’re looking for, read the paper to make sure. Also: sometimes you’ll read a paper that cites the paper we’re looking for. In such cases, please go to the references in order to find the full name of the paper we’re looking for, and search for it, and then select it. Significant progress was made by [CITATION], who gave a characterization of proper strong computable PAC learning in terms of the computability of a Empirical Risk Minimizer (ERM) and who constructed a class of finite VC dimension which is not computable PAC learnable, even in the improper sense.

- Paper ID: 4d666d422b9654c19deecf66c3789096e4c70ce0 Title: Computable PAC Learning of Continuous Features Abstract: We introduce defi... - Paper ID: 4d666d422b9654c19deecf66c3789096e4c70ce0 Title: Computable PAC Learning of Continuous Features Abstract: We introduce definitions of computable PAC learning for binary classification over computable metric spaces. We provide sufficient conditions on a hypothesis class to ensure than an empirical risk minimizer (ERM) is computable, and bound the strong Weihrauch degree of an ERM under more general conditions. We also give a presentation of a hypothesis class that does not admit any proper computable PAC learner with computable sample function, despite the underlying class being PAC learnable. Citation Count: 2 - Paper ID: d6cba2c3d7045ca34cd09a4f343860b25e4e3e2b Title: On computable learning of continuous features Abstract: We introduce definitions of computable PAC learning for binary classification over computable metric spaces. We provide sufficient conditions for learners that are empirical risk minimizers (ERM) to be computable, and bound the strong Weih... - Paper ID: 4d666d422b9654c19deecf66c3789096e4c70ce0 Title: Computable PAC Learning of Continuous Features Abstract: We introduce definitions of computable PAC learning for binary classification over computable metric spaces. We provide sufficient conditions on a hypothesis class to ensure than an empirical risk minimizer (ERM) is computable, and bound the strong Weihrauch degree of an ERM under more general conditions. We also give a presentation of a hypothesis class that does not admit any proper computable PAC learner with computable sample function, despite the underlying class being PAC learnable. Citation Count: 2 - Paper ID: d6cba2c3d7045ca34cd09a4f343860b25e4e3e2b Title: On computable learning of continuous features Abstract: We introduce definitions of computable PAC learning for binary classification over computable metric spaces. We provide sufficient conditions for learners that are empirical risk minimizers (ERM) to be computable, and bound the strong Weihrauch degree of an ERM learner under more general conditions. We also give a presentation of a hypothesis class that does not admit any proper computable PAC learner with computable sample function, despite the underlying class being PAC learnable. Citation Count: 2 - Paper ID: 970e7f3e4db249143b2957713bcb21bcbeac7675 Title: Probabilistically Robust PAC Learning Abstract: Recently, Robey et al. propose a notion of probabilistic robustness, which, at a high-level, requires a classifier to be robust to most but not all perturbations. They show that for certain hypothesis classes where proper learning under worst-case robustness is not possible, proper learning under probabilistic robustness is possible with sample complexity exponentially smaller than in the worst-case robustness setting. This motivates the question of whether proper learning under probabilistic robustness is always possible. In this paper, we show that this is not the case. We exhibit examples of hypothesis classes H with finite VC dimension that are not probabilistically robustly PAC learnable with any proper learning rule. However, if we compare the output of the learner to the best hypothesis for a slightly stronger level of probabilistic robustness, we show that not only is proper learning always possible, but it is possible via empirical risk minimization Citation Count: 1 - Paper ID: bd18733c4006516778048da64b35030ad5dfcdbe Title: Universality of empirical risk minimization Abstract: Consider supervised learning from i.i.d. samples $\{{\boldsymbol x}_i,y_i\}_{i\le n}$ where ${\boldsymbol x}_i \in\mathbb{R}^p$ are feature vectors and ${y} \in \mathbb{R}$ are labels. We study empirical risk minimization over a class of functions that are parameterized by $\mathsf{k} = O(1)$ vectors ${\boldsymbol \theta}_1, . . . , {\boldsymbol \theta}_{\mathsf k} \in \mathbb{R}^p$ , and prove universality results both for the training and test error. Namely, under the proportional asymptotics $n,p\to\infty$, with $n/p = \Theta(1)$, we prove that the training error depends on the random features distribution only through its covariance structure. Further, we prove that the minimum test error over near-empirical risk minimizers enjoys similar universality properties. In particular, the asymptotics of these quantities can be computed $-$to leading order$-$ under a simpler model in which the feature vectors ${\boldsymbol x}_i$ are replaced by Gaussian vectors ${\boldsymbol g}_i$ with the same covariance. Earlier universality results were limited to strongly convex learning procedures, or to feature vectors ${\boldsymbol x}_i$ with independent entries. Our results do not make any of these assumptions. Our assumptions are general enough to include feature vectors ${\boldsymbol x}_i$ that are produced by randomized featurization maps. In particular we explicitly check the assumptions for certain random features models (computing the output of a one-layer neural network with random weights) and neural tangent models (first-order Taylor approximation of two-layer networks). Citation Count: 45 - Paper ID: f48e41c4adf8d411a4fb7edea9176329c7d4fbfe Title: Proper Learning, Helly Number, and an Optimal SVM Bound Abstract: The classical PAC sample complexity bounds are stated for any Empirical Risk Minimizer (ERM) and contain an extra logarithmic factor $\log(1/{\epsilon})$ which is known to be necessary for ERM in general. It has been recently shown by Hanneke (2016) that the optimal sample complexity of PAC learning for any VC class C is achieved by a particular improper learning algorithm, which outputs a specific majority-vote of hypotheses in C. This leaves the question of when this bound can be achieved by proper learning algorithms, which are restricted to always output a hypothesis from C. In this paper we aim to characterize the classes for which the optimal sample complexity can be achieved by a proper learning algorithm. We identify that these classes can be characterized by the dual Helly number, which is a combinatorial parameter that arises in discrete geometry and abstract convexity. In particular, under general conditions on C, we show that the dual Helly number is bounded if and only if there is a proper learner that obtains the optimal joint dependence on $\epsilon$ and $\delta$. As further implications of our techniques we resolve a long-standing open problem posed by Vapnik and Chervonenkis (1974) on the performance of the Support Vector Machine by proving that the sample complexity of SVM in the realizable case is $\Theta((n/{\epsilon})+(1/{\epsilon})\log(1/{\delta}))$, where $n$ is the dimension. This gives the first optimal PAC bound for Halfspaces achieved by a proper learning algorithm, and moreover is computationally efficient. Citation Count: 38 - Paper ID: 9b6c4ce10e79d25270dbee8623f6819068ca15b6 Title: On Proper Learnability between Average- and Worst-case Robustness Abstract: Recently, Montasser et al. [2019] showed that finite VC dimension is not sufficient for proper adversarially robust PAC learning. In light of this hardness, there is a growing effort to study what type of relaxations to the adversarially robust PAC learning setup can enable proper learnability. In this work, we initiate the study of proper learning under relaxations of the worst-case robust loss. We give a family of robust loss relaxations under which VC classes are properly PAC learnable with sample complexity close to what one would require in the standard PAC learning setup. On the other hand, we show that for an existing and natural relaxation of the worst-case robust loss, finite VC dimension is not sufficient for proper learning. Lastly, we give new generalization guarantees for the adversarially robust empirical risk minimizer. Citation Count: 1 - Paper ID: 0ec03a7d5cd5d9c3d8e723dfb088f6afa970a299 Title: Weighted Empirical Risk Minimization: Sample Selection Bias Correction based on Importance Sampling Abstract: We consider statistical learning problems, when the distribution $P'$ of the training observations $Z'_1,\; \ldots,\; Z'_n$ differs from the distribution $P$ involved in the risk one seeks to minimize (referred to as the test distribution) but is still defined on the same measurable space as $P$ and dominates it. In the unrealistic case where the likelihood ratio $\Phi(z)=dP/dP'(z)$ is known, one may straightforwardly extends the Empirical Risk Minimization (ERM) approach to this specific transfer learning setup using the same idea as that behind Importance Sampling, by minimizing a weighted version of the empirical risk functional computed from the 'biased' training data $Z'_i$ with weights $\Phi(Z'_i)$. Although the importance function $\Phi(z)$ is generally unknown in practice, we show that, in various situations frequently encountered in practice, it takes a simple form and can be directly estimated from the $Z'_i$'s and some auxiliary information on the statistical population $P$. By means of linearization techniques, we then prove that the generalization capacity of the approach aforementioned is preserved when plugging the resulting estimates of the $\Phi(Z'_i)$'s into the weighted empirical risk. Beyond these theoretical guarantees, numerical results provide strong empirical evidence of the relevance of the approach promoted in this article. Citation Count: 10 - Paper ID: 1807fa82a2e28ff9ea144abaeee69fa90df3b397 Title: Revisiting generalization for deep learning: PAC-Bayes, flat minima, and generative models Abstract: In this work, we construct generalization bounds to understand existing learning algorithms and propose new ones. Generalization bounds relate empirical performance to future expected performance. The tightness of these bounds vary widely, and depends on the complexity of the learning task and the amount of data available, but also on how much information the bounds take into consideration. We are particularly concerned with data and algorithmdependent bounds that are quantitatively nonvacuous. We begin with an analysis of stochastic gradient descent (SGD) in supervised learning. By formalizing the notion of flat minima using PAC-Bayes generalization bounds, we obtain nonvacuous generalization bounds for stochastic classifiers based on SGD solutions. Despite strong empirical performance in many settings, SGD rapidly overfits in others. By combining nonvacuous generalization bounds and structural risk minimization, we arrive at an algorithm that trades-off accuracy and generalization guarantees. We also study generalization in the context of unsupervised learning. We propose to use a two sample test statistic for training neural network generator models and bound the gap between the population and the empirical estimate of the statistic. Citation Count: 8 - Paper ID: 45d849c6057660d801968d695a3bf518eacd58d6 Title: On the Fine-Grained Complexity of Empirical Risk Minimization: Kernel Methods and Neural Networks Abstract: Empirical risk minimization (ERM) is ubiquitous in machine learning and underlies most supervised learning methods. While there is a large body of work on algorithms for various ERM problems, the exact computational complexity of ERM is still not understood. We address this issue for multiple popular ERM problems including kernel SVMs, kernel ridge regression, and training the final layer of a neural network. In particular, we give conditional hardness results for these problems based on complexity-theoretic assumptions such as the Strong Exponential Time Hypothesis. Under these assumptions, we show that there are no algorithms that solve the aforementioned ERM problems to high accuracy in sub-quadratic time. We also give similar hardness results for computing the gradient of the empirical loss, which is the main computational burden in many non-convex learning tasks. Citation Count: 35 - Paper ID: 29a70dc873e66eb90e1aec2804ebe188f9158058 Title: Efficient Distributed Hessian Free Algorithm for Large-scale Empirical Risk Minimization via Accumulating Sample Strategy Abstract: In this paper, we propose a Distributed Accumulated Newton Conjugate gradiEnt (DANCE) method in which sample size is gradually increasing to quickly obtain a solution whose empirical loss is under satisfactory statistical accuracy. Our proposed method is multistage in which the solution of a stage serves as a warm start for the next stage which contains more samples (including the samples in the previous stage). The proposed multistage algorithm reduces the number of passes over data to achieve the statistical accuracy of the full training set. Moreover, our algorithm in nature is easy to be distributed and shares the strong scaling property indicating that acceleration is always expected by using more computing nodes. Various iteration complexity results regarding descent direction computation, communication efficiency and stopping criteria are analyzed under convex setting. Our numerical results illustrate that the proposed method outperforms other comparable methods for solving learning problems including neural networks. Citation Count: 17

There was an error reading the PDF. Please try a different paper. There was an error reading the PDF. Please try a different paper. There was an error reading the PDF. Please try a different paper.

arXiv:2111.14630v1 [cs.LG] 24 Nov 2021ON COMPUTABLE LEARNING OF CONTINUOUS FEATURES NATHANAEL ACKERMAN, JULIAN ASILIS, JIEQI DI, CAMERON F... arXiv:2111.14630v1 [cs.LG] 24 Nov 2021ON COMPUTABLE LEARNING OF CONTINUOUS FEATURES NATHANAEL ACKERMAN, JULIAN ASILIS, JIEQI DI, CAMERON FREER , AND JEAN-BAPTISTE TRISTAN Abstract. We introduce definitions of computable PAC learning for binary classification over com- putable metric spaces. We provide sufficient conditions for l earners that are empirical risk minimizers (ERM) to be computable, and bound the strong Weihrauch degre e of an ERM learner under more general conditions. We also give a presentation of a hypothe sis class that does not admit any proper computable PAC learner with computable sample function, de spite the underlying class being PAC learnable. Contents 1. Introduction 1 1.1. Related work 2 2. Preliminaries 3 2.1. Computable metric spaces and Weihrauch reducibility 3 2.2. Learning theory 6 3. Notions of computable learning theory 8 3.1. Countable hypothesis classes 9 3.2. Examples 10 3.3. Computable learners with noncomputable sample functi ons 11 4. Computability o... arXiv:2111.14630v1 [cs.LG] 24 Nov 2021ON COMPUTABLE LEARNING OF CONTINUOUS FEATURES NATHANAEL ACKERMAN, JULIAN ASILIS, JIEQI DI, CAMERON FREER , AND JEAN-BAPTISTE TRISTAN Abstract. We introduce definitions of computable PAC learning for binary classification over com- putable metric spaces. We provide sufficient conditions for l earners that are empirical risk minimizers (ERM) to be computable, and bound the strong Weihrauch degre e of an ERM learner under more general conditions. We also give a presentation of a hypothe sis class that does not admit any proper computable PAC learner with computable sample function, de spite the underlying class being PAC learnable. Contents 1. Introduction 1 1.1. Related work 2 2. Preliminaries 3 2.1. Computable metric spaces and Weihrauch reducibility 3 2.2. Learning theory 6 3. Notions of computable learning theory 8 3.1. Countable hypothesis classes 9 3.2. Examples 10 3.3. Computable learners with noncomputable sample functi ons 11 4. Computability of learners 12 4.1. Upper bounds 12 4.2. Lower bounds 13 Acknowledgements 16 References 16 1.Introduction The modern statistical learning theory framework for the st udy of uniform learnability is the synthesis of two theories. On the one hand, Vapnik–Chervonenkis (VC) theory [VC71 ] is a statistical theory that provides a rate of convergence for a uniform law o f large numbers for estimates of the form 1 n/summationtextn i=1I(f(Xi)/\e}atio\slash=Yi), where ( Xi, Yi) are i.i.d. samples from an unknown probability measure ove r X × Y andf:X → Y is a function from a class Hof measurable functions. The rate of convergence is a function of the complexity of the class H, measured using the concept of VC dimension . On the other hand, efficient Probably Approximately Correct (PAC) learnabilit y[Val84 ] is a computational theory that defines the efficient learnability of a function cl assHin terms of the existence of a learner , given by an algorithm having polynomial runtime, that takes an i.i.d. sample S=/parenleftbig (Xi, Yi)/parenrightbig i<nfrom an unknown probability measure µas input and returns a function h∈ H whose error Pr( h(X)/\e}atio\slash=Y) for (X, Y )∼µcan be bounded with high probability over the choice of S. The analogous notion ofPAC learnability , where the learner is merely required to be measurable in an appropriate sense, rather than efficiently computable, has also been widely stud ied. The synthesis of these two theories culminates with the so-c alled fundamental theorem of machine learning [ BEHW89 ], which establishes, under certain broadly-applicable me asurability conditions, that 1ON COMPUTABLE LEARNING OF CONTINUOUS FEATURES 2 a class of functions is PAC learnable if and only if its VC dime nsion is finite. This theory provides a justification for the foundational learning paradigm of em pirical risk minimization and has become the basis for studying many other learning paradigms and non -uniform theories of learnability. Note, however, that in this framework the learner is only required to be a measurable function, and in particular need not be computable. Insofar as the goal of studying uniform learning is to determ ine when a problem admits supervised learning by some program given access to training examples, it is important to investigate the subclass of learners that are in some sense computable , a natural object of study intermediate between learners that are efficiently computable and those that are merely meas urable. In this direction, [ AAB+20] proposed a notion of computable learner for computably repr esented hypothesis classes Hon discrete spaces. They principally consider binary classification in the case where His a computably enumerable set of computable functions on a countable domain, e.g., X=N. However, many natural problems considered in classical PAC learning theory have continuous domains, such as Rn. In the present paper, we consider notions of computable lea rners and hypothesis classes, without restricting to the discrete setting, e.g. , where Xis an arbitrary computable metric space. We do so using the framework of computable analysis [ Wei00 ], and establish upper and lower bounds on the computability of several standard classes of l earners in our setting. We now describe the structure of the paper. Next, in Section 1.1, we describe several other ap- proaches to computability in learning theory, including [ AAB+20], and their relation to our work. We then in Section 2provide the relevant preliminaries from computability the ory (including computable metric spaces and Weihrauch reducibility) and from classic al PAC learning theory. In Section 3we develop the basic concepts of computable learning theory in our setting, including notions of com- putability for learners, presentations of hypothesis clas ses, and sample functions. Section 4contains our primary results, including sufficient conditions for emp irical risk minimizer (ERM) learners to be computable, upper bounds on the strong Weihrauch degrees of certain ERM learners, and the construction of a (computable presentation of a) hypothesi s class that is PAC learnable but which has no computable proper PAC learner that admits a computable sa mple function. 1.1.Related work. Computability of PAC learners has also been studied in [ AAB+20], which con- siders the setting of discrete features and countable hypothesis classes. They provide several positive and negative results on the computability of both proper and improper learners for various notions of computably presented hypothesis classes, in both the rea lizable and agnostic cases. Our results, when we restrict our setting to discrete spaces, correspond most closely to their results for so-called recursively enumerably representable (RER) hypothesis classes. In particular, our Theorem 4.2can be viewed as a generalization of [ AAB+20, Theorem 10], and the proof of our Theorem 4.6uses similar ideas to those in [ AAB+20, Theorem 11]. Computability of non-uniform learning, which we do not consider in this paper, has been stu died in the discrete setting in both [ Sol08 ] and [ AAB+20]. In the present paper (and [ AAB+20]) when considering a function with finite codomain (as arise s for both learners and presentations of hypothesis classes) , the notion of computable function is such that for each input, the output is always eventually given. I t is also reasonable to consider settings in which there is a particular value signaling non-halting, which the computable function may never identify. This approach is explored in [ CMPR21 ], where non-halting of a learner’s output is signaled by the value ⊥. A related approach is considered in [ Cal15 ], which studies PAC learning for concepts that are Π0 1classes on 2N, which can be thought of as equivalent to working with comput able functions from 2Nto Sierpi´ nski space S(i.e., the space {⊥,⊤}with open sets {∅,{⊤},{⊥,⊤}}), where the inverse image of ⊤is the Π0 1class in question. Another interaction between learning theory and computabi lity is in the setting of “learning in the limit” [ Gol67 ], sometimes called TxtEx learning . One recent result [ Ber14 ] in this framework establishes the Σ0 3-completeness of this learning problem for certain computa bly enumerable hypothesis classes.ON COMPUTABLE LEARNING OF CONTINUOUS FEATURES 3 2.Preliminaries This section provides a brief treatment of the computabilit y theory and classical learning theory that form the starting point of our study. We begin by recalling several pieces of notation. For a set I, we write ( si)i∈Ito denote an I- indexed sequence. For n∈N, write [ n] to denote the set {0,1, . . . , n −1}. We write f↾Uto denote the restriction of a function f:X→Yto a subdomain U⊆X. For a topological space X, we write X<ωfor the space/coproducttext i∈NXiof finite sequences of points in X, endowed with its natural topology as the coproduct of produc t spaces. An extended metric space is a set Xequipped with a distance function d:X×X→R∪ {∞} satisfying the usual metric axioms (where ∞+r=∞for any r∈R∪ {∞} ). 2.1.Computable metric spaces and Weihrauch reducibility. We next describe certain key notions of computability and computable analysis, includi ng the notions of computable metric spaces and computable functions between them. For more details and several equivalent formulations of the basic notions, see, e.g., [ BHW08 , Section 4]. We then describe the notion of Weihrauch reduci bility; for more details, see [ BGP21 ]. Recall that a partial function ffromNtoNis said to be computable if there is some Turing machine that halts on input n(encoded in binary) precisely when fis defined on n, and in this case produces (a binary encoding of) f(n) as output. We fix a standard encoding of Turing machines and write {e}to denote the partial function that the program encoded by e∈Nrepresents. We write {e}(n)↓to mean that the partial function {e}is defined on n, i.e., that the program encoded by e halts on input n, and write {e}(n)↑otherwise. In this paper, it will be convenient to take oracles to be elem ents ofNNrather than 2N. For f∈NN we write {e}fto denote the partial function defined by an oracle program en coded by eusing fas an oracle. Because we are using oracles in NN, we will define the Turing jump to yield a function rather than a set. Given f∈NN, the Turing jump off, written f′, is defined to be the characteristic function of {e∈N:{e}f(0)↓}. By convention, we write ∅′for the characteristic function of the halting set {e∈N:{e}(0)↓}. A subset of Niscomputable if its characteristic function is a total computable functi on, and is computably enumerable (c.e.) if it is the domain of a partial computable function (e quivalently, either empty or the range of a total computable function). We will also speak of more elaborate finitary objects (such as sets of finite tuples of rationals) a s being computable or c.e. when they are computable or c.e., respectively, under a standard encodin g of the objects via natural numbers. For concreteness, we will use the notion of a presentation of a real when defining computable metric spaces, but note that this could also be formulated using rep resented spaces, as defined later in the section. An extended real is an element of R∪ {∞} . A presentation of an extended real is a sequence of rationals ( qi)i∈Nwith either qi> ifor all ior|qi−qj|<2−ifor all i,jwith i < j . In the first case we say that the sequence is a presentation of ∞and in the second case that it is a presentation of the limit of the Cauchy sequence in R. We say that an extended real is computable if it has a computable presentation. The computable reals are the elements of Radmitting a computable presentation as extended reals. We say that a sequence ( ti)i∈Nin an (extended) metric space X= (X, d) is a rapidly converging Cauchy sequence when for all i < j we have d(ti, tj)<2−i. Definition 2.1. Acomputable (extended) metric space is a triple X= (X, dX,(sX i)i∈N) such that (1) ( X∪S, dX) is a separable (extended) metric space, where S={sX:i∈N}, (2) ( sX i)i∈N, called the sequence of ideal points of X, enumerates a dense subset of ( X∪S, dX), (3)X, called the underlying set ofX, is dense in ( X∪S, dX), and (4)dX, called the distance function , is such that dX(sX i, sX j) is a computable extended real, uniformly in iandj.ON COMPUTABLE LEARNING OF CONTINUOUS FEATURES 4 In the special case where ( X, dX) is a complete (extended) metric space, we say that Xis acomputable (extended) Polish space . An element x∈Xis said to be a computable point ofXif there is a computable function f:N→Nsuch that ( sX f(i))i∈Nis a rapidly converging Cauchy sequence that converges to x. We will omit the superscripts and subscripts when they are c lear from context. Note that some papers (e.g., [ BP03 , Definition 2.1] and [ BHW08 , Definition 7.1]) define a com- putable metric space only in the case where the set Sof ideal points is a subset of X, and others (e.g., [HR09 , Definition 2.4.1]) use the term computable metric space to r efer to what we call a computable Polish space. Example 2.2. The setRof real numbers forms a computable Polish space under the Euc lidean metric, when equipped with the set Qof rationals as ideal points under the standard diagonal enu meration (qi)i∈N. The computable points of this computable Polish space are p recisely the computable reals. Note that in a computable metric space that is not a Polish spa ce, the ideal points need not be in the underlying set, as in the following example. Example 2.3. The set of irrational numbers forms a computable metric spac e under the Euclidean metric, again equipped ( qi)i∈Nas the sequence of ideal points. The computable points of thi s com- putable metric space are the computable irrational numbers . The next two examples will be key in many of our constructions . Example 2.4. Baire space , written NN, is the computable Polish space consisting of countably infi nite sequences of natural numbers, with ideal points those seque nces having only finitely many nonzero values (ordered lexicographically), and where dNNis the ultrametric on the countably infinite product of{0,1}, i.e., dNN/parenleftbig (si)i∈N,(ti)i∈N/parenrightbig = 2−infi∈N(si/negationslash=ti). Cantor space , written 2N, is the computable Polish subspace of NNconsisting of binary sequences. Letπ0andπ1be computable maps from NtoNsuch that i/mapsto→(π0(i), π1(i)) is a computable bijection of NwithN×N. WhenXandYare computable (extended) metric spaces, we write X×Yto denote the computable (extended) metric spaces with underlying set X×Y, with sequence of ideal points/parenleftbig (sX π0(i), sY π1(i))/parenrightbig i∈N, and where/parenleftbig (X∪SX)×(Y∪SY), dX×Y/parenrightbig is the product (extended) metric space of/parenleftbig X∪SX, dX/parenrightbig and/parenleftbig Y∪SY, dY/parenrightbig . We letX<ωbe the coproduct/coproducttext n∈ω/producttext i∈[n]X, i.e., the space whose underlying set consists of finite sequences of elements of X, whose ideal points are finite sequences of ideal points in X, and where the distance function satisfies dX<ω/parenleftbig (xi)i∈[n],(yi)i∈[m]/parenrightbig =/braceleftBigg max i∈[n]dX(xi, yi) if m=n; ∞ otherwise . Definition 2.5. Suppose X= (X, d X) and Y= (Y, d Y) are metric spaces and Z⊆X. We say a map f:X→Yiscontinuous onZif for all open sets U⊆Y, there is an open set V⊆Xsuch that f−1(U)∩Z=V∩Z. In other words, frestricted to Zis continuous as a map from the metric space that Xinduces on ZtoY. Definition 2.6. LetXandYbe computable metric spaces with ideal points ( si)i∈Nand ( ti)i∈N respectively, and suppose Z⊆X. Suppose f:W→Yis a map where Z⊆W⊆X. We say that f iscomputable on Zif for all ( j, q)∈N×Qthere is a set Φ j,q⊆N×Qsuch that •f−1(B(tj, q))∩Z=/parenleftbig/uniontext (k,p)∈Φj,qB(sk, p)/parenrightbig ∩Z, and •the set {(j, q, k, p ) : ( k, p)∈Φj,q}is c.e. This definition captures the notion that the partial map fis continuous on its restriction to Zand has a computable witness to this continuity.ON COMPUTABLE LEARNING OF CONTINUOUS FEATURES 5 Observe that a computable function from NNto a computable metric space Ycan be thought of as a program on an oracle Turing machine that takes the input o n its oracle tape, and outputs a “representation” of a point in Y. The notion of a represented space is one way of making this notion precise. For more details, see [ BGP21 ]. Definition 2.7. Arepresented space (X, γ) is a set Xalong with a surjection γfrom a subset of NNonto X. When the choice of γis clear from context, we call γtherepresentation ofX. Definition 2.8. Suppose X= (X, dX,(sX i)i∈N) is a computable metric space. Define CS X⊆NNto be the collection of functions f:N→Nfor which/parenleftbig sX f(i)/parenrightbig i∈Nis a rapidly converging Cauchy sequence whose limit is in X. The represented space induced by Xis defined to be ( X, γX), where γX: CSX→X assigns each function fthe value lim i→∞sX f(i). Intuitively, a realizer of a function gtakes a description of an input xto a description of the corresponding output g(x), where these descriptions are given in terms of representa tions. Definition 2.9. Suppose ( X, γ X) and ( Y, γ Y) are represented spaces, and let g:X→Ybe a map. Arealizer ofgis any function G: dom( γX)→dom( γY) such that γY◦G=g◦γX. A realizer is computable if it is computable on dom( γX) (considered as a partial map between computable metric spaces NNandNN). The notion of strong Weihrauch reducibility aims to capture the intuitive idea that one function is computable given the other function as an oracle, along with possibly some computable pre-processing and post-processing, where access to the original input is p ermitted only in pre-processing. (The weaker notion of Weihrauch reducibility , in which the input may be used again in post-processing, als o arises in computable analysis, but in this paper we are able t o show that all of the relevant reductions are strong.) Definition 2.10. Let ( Xi, γXi) and ( Yi, γYi) be represented spaces for i∈ {0,1}, and suppose that f:X0→Y0andg:X1→Y1are functions. Let FandGbe the sets of realizers of fandgrespectively. We say that fisstrongly Weihrauch reducible tog, and write f≤sWg, when there are computable functions HandK, each from some subset of NNtoNN, such that for every G∈ Gthere exists an F∈ Fsatisfying F=H◦G◦K. We say that fandgarestrongly Weihrauch equivalent , and write f≡sWg, when f≤sWgand g≤sWf. Note that strong Weihrauch reducibility is usually describ ed in the more general setting of partial multifunctions. Here we will only need single-valued funct ions with explicitly defined domains, and Definition 2.10 coincides with the standard one in this situation. The following important map describes the problem of comput ing limits on a represented space Xinduced by a computable metric space X. (Note that elsewhere in the literature, lim Xis typically referred to as lim X.) Definition 2.11. Suppose Xis a computable metric space, and let ( X, γX) be the represented space it induces. The limit map limX:XN→Xis the function that assigns every convergent Cauchy sequence in Xits limit. One can view lim NNas playing a role in Weihrauch reducibility analogous to the role played by the halting problem ∅′with respect to Turing reducibility. For more details, see [ BGP21 , §11.6]. It will also be useful to introduce the notion of a rich space, which bears a relation to lim NNand is informally a space that computably contains the real number s. Definition 2.12. A computable metric space Xisrich if there is some computable map ι: 2N→X that is injective and whose partial inverse ι−1is also injective.ON COMPUTABLE LEARNING OF CONTINUOUS FEATURES 6 Lemma 2.13. [BGP21 , Proposition 11.6.2] IfXandYare rich spaces, then limX≡sWlimY. In particular, limX≡sWlimNN. Observe that for any computable metric space X, the space X/coproducttextNNis rich, and therefore lim X≤sW limX/coproducttext NN≡sWlimNN. Hence lim NNis maximal (under ≤sW) among limit operators. We will also work with the Turing jump map J:NN→NN, given by z/mapsto→z′, which is strongly Weihrauch equivalent to lim NN. Lemma 2.14 ([BGP21 , Theorem 11.6.7]) .limNN≡sWJ. Although lim NN≡sWJ, in general lim Iis weaker. In Section 4.1we will establish our upper bounds in terms of lim Ifor appropriate computable metric spaces I, while in Section 4.2we will establish a bound using the operator J. Strong Weihrauch reductions to the parallelization of a function allow one to ask for countably many instances of the function to be evaluated. Definition 2.15. Letf:X→Ybe a map between represented spaces. The parallelization offis the map /hatwidef:XN→YNdefined by /hatwidef/parenleftbig (xi)i∈N/parenrightbig =/parenleftbig f(xi)/parenrightbig i∈N. The following is immediate. Lemma 2.16. For any map f:X→Ybetween represented spaces, f≤sW/hatwidef. We will also need the following standard fact. Lemma 2.17 ([BGP21 , Theorem 11.6.6]) ./hatwiderlimNN≡sWlimNN. The notion of the parallelization of a function will be impor tant in Section 4.2, for reasons we explain in Remark 4.7. 2.2.Learning theory. We now consider the traditional framework for uniform learn ability, formu- lated for Borel measurable hypotheses. A learning problem i s determined by a domain, label set, and hypothesis class, as we now describe. (i) a domain Xof features that is a Borel subset of some extended Polish spa ceX, (ii) a label set Ythat is a Polish space, and (iii) a hypothesis class Hconsisting of Borel functions from XtoY. We will say that any Borel function from XtoYis ahypothesis ; note that such a map is sometimes also called a predictor ,classifier , orconcept . In this paper, we will only consider problems in binary classification, i.e., where Y={0,1}, considered as a metric space under the discrete topology. LetDbe a Borel measure on X × Y . The true error , or simply error , of a hypothesis h∈ Hwith respect to Dis the probability that ( x, h(x)) disagrees with a randomly selected pair drawn from D, i.e., LD(h) =D/parenleftbig {(x, y)∈ X × Y | y/\e}atio\slash=h(x)}/parenrightbig . Theempirical error of a hypothesis hon a tuple S= ((x1, y1), . . . , (xn, yn))∈(X ×Y )noftraining examples is the fraction of pairs in Son which hmisclassifies the label of a feature, i.e., LS(h) =/summationtextn i=1|h(xi)−yi| n. Traditionally, one thinks of a learner as a map which takes fin ite sequences of ( X ×Y )<ωand returns a hypothesis, i.e., an element of YX. We would then like to define a computable learner as a learner which is computable as a map between computable extended met ric spaces. Unfortunately, here we encounter the obstructions that YXis not, in general, an extended metric space. We overcome it b y instead considering a learner as the “curried” version of a m ap from ( X × Y )<ωtoYX, i.e., as a map from ( X × Y )<ω× X → Y . In this manner, we will be able to consider learners which ar e computable as maps between Polish spaces.ON COMPUTABLE LEARNING OF CONTINUOUS FEATURES 7 Definition 2.18. Alearner is a Borel measurable function A: (X × Y )<ω× X → Y . For notational convenience, for S∈(X × Y )<ωwe let /tildewideA(S):X → Y be the function defined by /tildewideA(S)(x) =A(S, x). The goal of a learner Ais to return a hypothesis hthat minimizes the true error with respect to an unknown Borel distribution DonX × Y . The learner does so by examining a D-i.i.d. sequence S=/parenleftbig (x1, y1), . . . , (xn, yn)/parenrightbig . Notably, the learner cannot directly evaluate LD; it is guided only by the information contained in the sample S, including evaluations of LS. However, as it is ignorant of D, the learner does not know how faithfully LSapproximates LD. The most central framework for assessing learners with resp ect to hypothesis classes is that of PAC learning (see, e.g., [ SB14 , Chapter 3]). In the setting of efficient PAC learning [ SB14 , Definition 8.1], one further requires that the learning algorithm be polynom ial-time in the reciprocal of its inputs ǫ andδ, to be described in the following definition. Definition 2.19. LetDbe a collection of Borel distributions on X × Y and let Hbe a hypothesis class. A learner Ais said to PAC learn Hwith respect to D(or is a learner for Hwith respect toD) if there exists a function m: (0,1)2→N, called a sample function , that is non-increasing on each coordinate and satisfies the following property: for ev eryǫ, δ∈(0,1) and every Borel distribution D ∈D, a finite i.i.d. sample Sfrom Dwith |S| ≥m(ǫ, δ) is such that, with probability at least (1 −δ) over the choice of S, the learner Aoutputs a hypothesis /tildewideA(S) with (†) LD(/tildewideA(S))≤inf h∈HLD(h) +ǫ. (Observe that ( †) is a Borel measurable condition, as LD(/tildewideA(S)) =/integraltext/BDA(S,x)/negationslash=yD(dx, dy ).) The minimal such sample function for Ais its sample complexity . When there is some learner Athat learns H with respect to D, we say that HisPAC learnable with respect to D(viaA). In the case where Dconsists of all Borel distributions on X × Y , we say that Hisagnostic PAC learnable and that Ais an agnostic PAC learner for H. In the case where Dconsists of the class of Borel distributions DonX × Y for which LD(h) = 0 for some h∈ H, we say that HisPAC learnable in the realizable case and that APAC learns Hin the realizable case . Remark 2.20. Some sources use “sample complexity” to refer to a property o f hypothesis classes H, defined as the pointwise minimum of all of H’s PAC learners’ sample complexities (in the sense of Definition 2.19). The learner-dependent definition will be more appropriat e for our purposes, in which, for instance, the distinction between computable and nonco mputable learners is of central importance. We will see shortly in Theorem 2.23 that a class that is PAC learnable in the realizable case must also be agnostic PAC learnable (possibly via a different lear ner with worse sample complexity). Definition 2.21. A learner Eis an empirical risk minimizer (orERM ) for H, if for all finite sequences S∈(X × Y )<ω, we have /tildewideE(S)∈argminh∈HLS(h). Definition 2.22. TheVC dimension ofHis sup/braceleftbig |C|:C⊆ X and {h↾C:h∈ H/bracerightbig ={0,1}C}. When {h↾C:h∈ H/bracerightbig ={0,1}C, we say that Hshatters the set C. We now state the relevant portions of the fundamental theore m of learning theory in our setting (binary classification with 0-1 loss), which holds for hypot hesis classes satisfying the mild technical assumption of universal separability [BEHW89 , Appendix A]. This condition is satisfied for any hy- pothesis class having a computable presentation (see Defini tion3.2), as is the case for all hypothesis classes considered in this paper.ON COMPUTABLE LEARNING OF CONTINUOUS FEATURES 8 Theorem 2.23 ([SB14 , Theorem 6.7]) .LetHbe a hypothesis class of functions from a domain Xto {0,1}. Then the following are equivalent: 1.Hhas finite VC dimension. 2.His PAC learnable in the realizable case. 3.His agnostically PAC learnable. 4. Any ERM learner is a PAC learner for H, over any family of measures. Because of the equivalence between conditions 2 and 3, we wil l say that a hypothesis class HisPAC learnable (without reference to a class of distributions D, and without mentioning agnostic learning or realizability) when any of these equivalent conditions h old. Note that while every agnostic PAC learner for His in particular a PAC learner for Hin the realizable case, the converse is not true; when we speak of a PAC learner for Hwithout mention of D, we will mean the strongest such instance, namely that it is an agnostic PAC learner for H. Furthermore, there exists a connection between the VC dimen sion of a PAC learnable class and the sample functions of its ERM learners. Theorem 2.24 ([SB14 , pp. 392]) .LetHbe a hypothesis class of functions from a domain Xto{0,1} with finite VC dimension d. Then its ERM learners are PAC learners with sample function s m(ǫ, δ) = 432d ǫ2·log/parenleftbigg64d ǫ2/parenrightbigg +8 ǫ2·(8dlog(ǫ/d) + 2 log(4 /δ)). 3.Notions of computable learning theory As described before Definition 2.18, the notion of learner we consider in this paper is the curried version of the standard one, in order to allow for it to be a com putable map between Polish spaces. We now make use of this, to define when a learner is computable, and when a hypothesis class is computably PAC learnable. Definition 3.1. By a computable learner we mean a learner A: (X × Y )<ω× X → Y which is computable as a map of computable extended metric spaces. We say a hypothesis class His computably PAC learnable if there is a computable learner that PAC learns it. It will also be important to have a computable handle on hypot hesis classes themselves. As such, we will primarily consider hypothesis classes as collectio n of hypotheses endowed with (not necessarily unique) indices. This information is collected up into a presentation of the class. Definition 3.2. Apresentation of a hypothesis class is a Borel measurable function H:I ×X → Y. We call Itheindex space . Let /tildewideH:I → YXbe the function defined by /tildewideH(i)(x) =H(i, x). We writeH†to denote the underlying hypothesis class, i.e., range( /tildewideH). We say that Hpresents the class H†and that a hypothesis is an element of Hwhen it is in H†. Definition 3.3. A presentation H:I × X → Y of a hypothesis class is computable ifIis a computable metric space and His computable as a map of computable extended metric spaces. Classically, a proper learner for a hypothesis class His usually regarded simply as a learner which happens to always produce hypotheses in the class H. This is a key notion, about which we will want to reason computably. In our setting, to study the computability of proper learnin g, it will be valuable to consider the case in which the elements of Hare identified by indices bearing additional structure, and thus to consider learners that identify hypotheses in Hby such indices, using a presentation H. Consequently, and in contrast to the classical setting, we take proper learners t o be slightly different objects than ordinary learners. Our proper learners map samples to indices, rathe r than map samples and features to labels. We then can define a computable proper learner to be simply a pr oper learner that is computable (similarly to Definition 3.1of a computable learner).ON COMPUTABLE LEARNING OF CONTINUOUS FEATURES 9 Definition 3.4. LetH:I × X → Y be a presentation of a hypothesis class. A proper learner for His a map A: (X × Y )<ω→ I. If the map Adefined by A((xi, yi)i∈[n], x) =H(A(xi, yi)i∈[n], x) is a PAC learner for H†, thenAis aproper PAC learner forH, and we call Athelearner induced by A(as a proper learner for H). IfHis a computable presentation, we say that a proper learner Afor Hiscomputable when it is computable as a map of computable extended metric s paces. Note that the learner Ainduced by a computable proper PAC learner for Hin Definition 3.4is a computable learner for H†, as we have required both AandHto be computable. Intuitively, His computably properly PAC learnable if there is a computable f unction which takes in finite sequences of elements of X × Y and outputs the index of an element of H, and where the corresponding learner PAC learns H†. Definition 3.5. Given a hypothesis class H, define Φ H⊆(X × Y )<ωto be the set of those finite sequences/parenleftbig (x1, y1), . . . , (xn, yn)/parenrightbig for which/braceleftbig (x1, y1), . . . , (xn, yn)/bracerightbig is a subset of the graph of hfor some h∈ H, i.e.,/uniontext h∈H/coproducttext n∈ω/braceleftbig (x, h(x)) : x∈ X/bracerightbign. Recall that the realizable case restricts attention to meas ures Dfor which D-i.i.d. sequences are almost surely in the graph of some element of H. In particular, for any such Dandn∈N, the product measure Dnis concentrated on Φ H∩(X × Y )n. Note, however, that Φ Hitself will not in general be Borel, even when His. Yet, in the following definition, Φ Hplays only the role of a subdomain on which the computability of learners in the realizable case i s considered, and thus its measure-theoretic properties are of no consequence. Definition 3.6. LetHbe a hypothesis class. Then a learner AforHin the realizable case is computable in the realizable case forHif it is computable on Φ H× X as a function between metric spaces ( X × Y )<ω× X andY. A proper learner Afor a computable presentation HofHis computable in the realizable case ifAis computable on Φ Has a function between metric spaces (X × Y )<ωandI. Note that it is possible to have a noncomputable learner for Hwhich is nevertheless computable in the realizable case for H. However, all computable learners for Hare computable in the realizable case for H. It will be important to impose computability constraints on sample functions as well as learners. Definition 3.7. A sample function m: (0,1)2→Niscomputable if uniformly in n∈Nthere are computable sequences of rationals ( ℓn,i)i∈N, (rn,i)i∈N, (tn,i)i∈N, and ( bn,i)i∈Nsuch that •Un⊆m−1(n) for every n∈N, and •the closure of the set/uniontext n∈NUnis (0,1)2, where for each nwe define Un=/uniontext i∈N(ℓn,i, rn,i)×(tn,i, bn,i). Given a computable PAC learner and a computable sample funct ion for this learner, one can produce an algorithm that, given an error rate and failure pr obability, outputs a hypothesis having at most that error rate with at most the stated failure probabil ity. If the computable learner is an ERM, then by Theorem 2.24 it has a computable sample function, and so one obtains such a n algorithm. On the other hand, we will see in Theorem 3.12 that not every computable PAC learner (for a given hypothesis class Hand class of distributions D) admits a computable sample function (with respect toHandD). 3.1.Countable hypothesis classes. Suppose that Xis countable and discrete. Requiring that a learner Abe computable is then tantamount to asking that the maps x/mapsto→A(S, x) be uniformly computable as Sranges over ( X × Y )<ω. By collecting up this data, such a computable learner Acan be encoded as a computable map from NtoN. In a similar fashion, a computable presentation of a hypothesis class could be encoded by a single computable map fromNtoN. The paper [ AAB+20] studies computable PAC learning in the setting where X=N, a countable discrete metric space. As such, they are able to work with the encodings of these simplified notions of computable learners and presentations of hypothesis cla sses, as we have just sketched.ON COMPUTABLE LEARNING OF CONTINUOUS FEATURES 10 3.2.Examples. To illustrate these definitions, we now describe two example s — one a very basic one in this formalism, and the other a standard example from l earning theory. 3.2.1. “Apply” function. Let the index space Ibe 2Nand the sample space XbeN. We define the “apply” presentation of the hypothesis class 2Nto be the map H:I ×X → { 0,1}whereH(x, n) =x(n). Note that while His computable, there is no single Turing degree which bounds every hypothesis in H†= 2N. In particular, this example demonstrates that the notion o f computable hypothesis class that we consider is fundamentally more general than the corr esponding notion in [ AAB+20], which considers only countable collections of hypotheses. 3.2.2. Decision stump. Recall the decision stump problem from classical learning t heory: X=R, Y={0,1}, and H={ /BD>c:c∈R}. In the realizable case, the learning problem amounts to estimating the true cutoff point cfrom a sample S= (xi, yi)i∈[n]for which yi= 1 if and only if xi> c. It is well-known to be PAC learnable in the realizable case vi a the following algorithm: 1. If Shas negatively labeled examples (i.e., ( xi, yi) with yi= 0), then set mto be the maximal such xi. Otherwise, set mto be the minimal feature among positively labeled examples . 2. Return /BD>m. In particular, this implements an ERM learner for Hin the realizable case. Further, as Hhas VC dimension 1, it is a PAC learner for Hin the realizable case by the equivalence of clauses 1 and 4 in Theorem 2.23. The classical algorithm does not give rise to a computable le arner in the sense of Definition 2.18, however, as /BD>mcannot be computed from S. In particular, undecidability of equality for real number s obstructs such a computation from being performed over R. In order to more sensibly cast the problem in a computable setting, we restrict focus to cutoff points lo cated at computable reals and take the noncomputable reals as the domain set X. Now consider the computable presentation Hstep:Rc×(R\Rc)→ {0,1}of a hypothesis class with index set the computable reals Rc, given by Hstep(c, x) = /BD>c(x). Its underlying hypothesis class H† step={ /BD>c:c∈Rc}consists of computable functions (whose domains are R\Rc), thus proper learners have a chance of success. Nevertheless, the classi cal algorithm fails: mwill reside in X, and thus /BD>mwill be noncomputable as a function on X(even when one has access to m). We will exhibit a proper learner AstepforHstepthat is computable in the realizable case and whose induced l earner is an ERM. Fix a computable enumeration ( qi)i∈NofQand uniformly enumerate a computable presentation of each as a computable real. Algorithm 3.8 (Algorithm Astep).Given a sample S= (xi, yi)i∈[n], output the least i∈Nfor which the empirical error of /BD>qiis 0. Proposition 3.9. Astepis a proper learner for Hstepthat is computable in the realizable case and whose induced learner is an ERM. Proof. Observe that the sequence of functions ( /BD>qi)i∈Nis uniformly computable on X=R\Rc. The empirical error of each /BD>qican be computed exactly on any sample (and hence compared wit h 0). The loop terminates upon reaching a rational qithat separates the sample S, one of which must exist for any Sunder consideration in the realizable case. /square Corollary 3.10. Astepis a computable proper PAC learner in the realizable case for Hstep. Proof. By Proposition 3.9,Astepis a computable proper learner in the realizable case for Hstep, whose induced learner is an ERM. The class H† stephas VC dimension 1, and so by the equivalence of clauses 1 and 4 in Theorem 2.23, the learner induced by Astepis a PAC learner in the realizable case. /square In fact, we will see shortly in Theorem 4.2that Corollary 3.10 is an instance of a more general result, namely that all classes with computable presentati ons have computable ERM learners in the realizable case.ON COMPUTABLE LEARNING OF CONTINUOUS FEATURES 11 3.3.Computable learners with noncomputable sample functions. Theorem 3.12 shows that even when a hypothesis class Hand class of distributions Dadmit some computable PAC learner with a computable sample function, not all computable learn ers for Hwith respect to Dmust have a computable sample function. Therefore, when investigating the computability of algori thms for outputting a hypothesis (with the desired error rate and failure probability), we must con sider the computability of a pair consisting of a PAC learner and sample function, not merely the PAC learn er alone. The intuition behind the proof of Theorem 3.12 is that we can enumerate those programs that halt, and whenever the nth program to halt does so, we then coarsen all samples of size nup to accuracy 2−s, where sis the size of the program. Consequently, for each desired de gree of accuracy, we eventually obtain answers that are never coarsened beyon d that accuracy. On the other hand, knowing how many samples are needed for a given accuracy allo ws us to determine a point past which we never again coarsen to a given level. This then lets us dedu ce when a given initial segment of the halting set has stabilized. Definition 3.11. ForM∈N, letDMbe the collection of Borel probability distributions Dover (R\Rc)× {0,1}such that (i)LD(h) = 0 for some element hofHstep, and (ii)Dis absolutely continuous (with respect to Lebesgue measure ) and has a probability density function bounded by M. Theorem 3.12. For each M∈N, there is a learner AonX=R\RcandY={0,1}such that •Ais computable in the realizable case with respect to H† step, •Ais a PAC learner for H† stepoverDM, and • ∅′is computable from any sample function for A(as a learner for H† stepoverDM). Proof. Define the function α:Q×N→Qbyα(q, ℓ) =⌊2ℓq⌋/2ℓ, and let c: (X × Y )<ω→Qbe such that c/parenleftbig {(xi, yi)}i∈[k]/parenrightbig is the rational qof least index attaining zero empirical error on {(xi, yi)}i∈[k]if one exists, and 0 otherwise. Hereafter, we will additionall y demand that the computable enumeration ofQemployed by cbe one which enumerates1⁄3first. Define c∗: (X × Y )<ω×N→Qbyc∗(S, n) = α(c(S), n), i.e., the previous decision stump learner discretized to accuracy 2−n. Let ( ek)k∈Nbe a computable enumeration without repetition of all e∈Nfor which {e}(0)↓. ForS∈(X × Y )<ω, write len( S) for its length. Define the function A: (X × Y )<ω× X → Y by A(S, x) =hc∗(S,e len( S))(x). In other words, we discretize the decision stump algorith m to accuracy 2−elen( S). Note that because lim kek=∞, we can find arbitrarily good approximations as we increase the sample size, even if (as we will show) we cannot compute ho w large such samples must be. Note that Ais computable in the realizable case. Further, for every r∈Nthere is an i∈Nsuch that er∗> ifor all r∗≥r. Then for every integer ℓ >0, there is an n∈Nsuch that whenever len(S)> n, the set U={x:Hstep(Astep(S), x)/\e}atio\slash=A(S, x)}is contained in an interval of length 2−ℓ. Ais thus a PAC learner for H† stepoverDM, as the loss incurred by AonUis bounded uniformly over DMby 2−ℓ·M. Letm(ǫ, δ) be a sample function for Aand consider n∈N. We will compute the function ∅′ restricted to the set [ n] ={0, . . . , n −1}. Fix any rational δ∈(0,1), and set mn=m(2−(n+2), δ). Suppose there is some i > m nsuch that ei< n. Then given a sample Sof size i, the function A(S,·) will discretize c(S) to an accuracy below 2−n. This would cause Ato incur a true loss of at least 2−non the distribution which is uniform on features in [0 ,1] and takes labels according to /BD>1/3, as α(1/3, k)≤1/3−2−(k+2), a contradiction. Hence i≤mnwhenever ei< n. We can therefore determine membership in {ek:k∈N} ∩[n], and hence can compute ∅′restricted to [ n]. /squareON COMPUTABLE LEARNING OF CONTINUOUS FEATURES 12 4.Computability of learners We now turn to the question of how computable a learner can be, for a hypothesis class with a computable presentation. Throughout this section, we remain in the setting of binary c lassification, i.e., Y={0,1}. 4.1.Upper bounds. For any computable presentation of a hypothesis class, we es tablish a concrete upper bound on the complexity of some ERM, which depends only on the index space. Theorem 4.1. Suppose H:I ×X → Y is a computable presentation of a hypothesis class. Then the re is an ERM for H†that is strongly Weihrauch reducible to limI. Proof. Fix a sample S= (xi, yi)i∈[n]. To invoke lim I, we introduce a procedure for approximating an input z= (zi)i∈N∈ IN. In particular, we approximate zusing the sequence ( zk)k∈N, with each zk∈ INtaking the form zk= (z1 k, . . . , zk−1 k, zk k, zk k, . . .), i.e., constant after the ( k−1)th term. zj kis computed as follows, for j∈[k]: 1. Take balls around the xiand around the first jideal points of I, all of radius 2−k. In addition, calculate which value is taken by yi∈ {0,1}. 2. For each of the first jideal points of I, useHto determine whether the balls around the xi and the ideal point suffice to calculate a well-defined empiric al error with respect to S. 3. If none of the first jideal points induce a well-defined empirical error, set zj kto be the first ideal point of I. Otherwise, set zj kto be the first ideal point which attains minimal empirical error among the first jideal points. AsHis continuous, and as there are only finitely many possible em pirical errors, if w∈ Iis such that /tildewideH(w) has minimal empirical error with respect to S, then there must be an open ball around wwhere all elements of the ball give rise to a function with the same minimal empirical error (with respect to S). In particular, there must be an ideal point csuch that /tildewideH(c) has minimal empirical error with respect to S. Therefore z= (zj j)j∈Nconverges to the ideal point with minimal index among those that give rise to minimal empirical error with respect toS. Calling lim Ionzthus is a proper learner whose induced learner for H†is an ERM, as desired. /square Furthermore, in this setting there is always an ERM that is co mputable in the realizable case. This result can be viewed as a generalization of [ AAB+20, Theorem 10]. Theorem 4.2. Suppose H:I ×X → Y is a computable presentation of a hypothesis class. Then the re is an ERM for H†that is computable in the realizable case. Proof. We will construct a proper learner whose induced learner for H†is an ERM that is computable in the realizable case. Suppose S∈ΦH†. There is some w∈ Isuch that /tildewideH(w) has empirical error 0 with respect to S. Because there are only finitely many possible values of the e mpirical error with respect to S, there must be some open ball Baround wsuch that for all elements w∗∈B, the function /tildewideH(w∗) has empirical error 0 with respect to S. In particular, there must be some ideal point cin this ball. Therefore the algorithm which searches through all id eal points and returns the first to attain an empirical error 0 with respect to Swill eventually halt. /square Remark 4.3. The algorithms in Theorems 4.1and4.2would have failed had Ynot been computably discrete, in which case verifying that a hypothesis incurs a n empirical error of 0 would not be com- putable. When Y={0,1}, as in this paper, the predictions of hypotheses on features can be deduced exactly, allowing for precise computation of empirical err ors. If Y=R, in contrast, then predictions of hypotheses htake the form ( qk−2−k, qk+ 2−k) for qk∈Qand chosen k∈N, amounting to the information that h(x)∈(qk−2−k, qk+ 2−k). Some such intervals allow one to conclude that h(x)/\e}atio\slash=y, namely when y /∈(qk−2−k, qk+ 2−k), and thus that hdoes not attain an empirical error of 0. Yet no such interval a llows one to conclude that h(x) =yfor even a single example ( x, y) ifymay take any real value, much less that hattains an empirical error of 0 across an entire sample.ON COMPUTABLE LEARNING OF CONTINUOUS FEATURES 13 The restricted setting of computability in the realizable c ase, as in Theorem 4.2, provides a stopping criterion for detecting a hypothesis in H†attaining minimal empirical risk on S, thereby eliminating the need for lim I. A similar criterion would arise if the size of the restricti on of a (computably presented) class Hto a given sample Scould be known in advance. In such a case, one could walk through the ideal points of Ias in 4.2until all such behaviors on Sare encountered, subsequently returning one which attains the minimal empirical error. Theorem 4.4. Suppose H:I × X → Y is a computable presentation of a hypothesis class, and that for all finite U⊆ X, the size of {h↾U:h∈H†}can be computed, uniformly in U. Then an ERM learner for H†is computable. Proof. Letn∈N. Define Hn:I × Xn→ Ynto be the map where Hn(w,(xj)j∈[n]) = (H(w, x j))j∈[n]. Note that this is a continuous function, and hence for all w∈ Iand for every u∈ Xnthere is an ideal point csuch that Hn(w, u) =Hn(c, u). Suppose U⊆ X is finite. We then have /vextendsingle/vextendsingle{h↾U:h∈H†)}/vextendsingle/vextendsingle=/vextendsingle/vextendsingle/braceleftbig/tildewideH(c)↾U:cis an ideal point of I/bracerightbig/vextendsingle/vextendsingle. In particular, by searching through all the ideal points of Iwe realize all behavior (restricted to U) that occurs in /tildewideH. So, from/vextendsingle/vextendsingle{h↾U:h∈H†}/vextendsingle/vextendsinglewe can compute ideal points of Irealizing all such behavior. From this it is straightforward to choose an ideal point which minimizes the empirical error on any sample ( xi, yi)i∈[m]where {xi:i∈[m]}=U. /square It has been shown in [ FW95 ] that the computability condition of Theorem 4.4is enjoyed by maxi- mum classes, i.e., those which achieve the bound of the Sauer –Shelah lemma. We can thus conclude computable PAC learnability for such maximum classes. Corollary 4.5. IfH:I × X → Y is a computable presentation of a hypothesis class, and H†is a maximum class of finite VC dimension, then it is computably PA C learnable. 4.2.Lower bounds. We now show that in appropriate circumstances, all proper le arners must have a certain complexity, thereby providing some correspondin g lower bounds. 4.2.1. Discrete index spaces. Suppose the index space of a computable presentation of a hyp othesis class is infinite and discrete (hence isomorphic to N). Theorem 4.1shows that there is an ERM that is strongly Weihrauch reducible to lim N. We now provide a partial converse in the same setting, showin g that there is a computable presen- tation of a hypothesis class with discrete index space such t hat∅′is strongly Weihrauch reducible to any proper PAC learner for the presentation along with any sa mple function for the proper learner. In particular, for this presentation, there is no computabl e procedure for outputting hypotheses from samples in a manner that PAC learns the underlying hypothesi s class. The hypothesis class that we will use in the proof of Theorem 4.6is similar to that used to prove [AAB+20, Theorem 11]. Theorem 4.6. There is a hypothesis class that is PAC learnable but admits a computable presentation Hwith discrete index space such that ∅′≤sW(A, m)whenever Ais a proper PAC learner for Hand mis a sample function for the learner induced by AforH†. Proof. Given an enumeration Z= (zi)i∈Nwithout repetition of some subset of N, define DZto be the computable metric space with underlying set {zi:i∈N}, the discrete metric taking distances {0,1}, and sequence of ideal points Z. LetE= (ei)i∈Nbe a computable enumeration without repetition of all natur al numbers esuch that {e}(0)↓. Given two natural numbers n0andn1, we write n0∼n1if either (a) both {n0}(0)↑ and{n1}(0)↑, or (b) both {n0}(0)↓and{n1}(0)↓and the programs n0andn1take the same number of steps to halt on input 0. Let the index space IbeDE, as defined above. Let the sample space XbeD(i)i∈N, and let H:I × X → { 0,1}be the map where H(n0, n1) = 1 if and only if n0∼n1. Note that His computable via the following algorithm:ON COMPUTABLE LEARNING OF CONTINUOUS FEATURES 14 (1) Run program n0on input 0 until it halts, which it must as n0∈ I. Let kbe the number of steps it took to halt. (2) Run program n1on input 0 for ksteps. (3) If n1takes precisely ksteps to halt on input 0 then return 1, and otherwise (i.e., if it takes fewer steps or has not yet halted) return 0. First we show that H†shatters no set of size 2, so that it has VC dimension 1 and henc e is PAC learnable by Theorem 2.23. Let n0, n1∈Nbe distinct. If there exists an h∈H†with h(n0) = 1 and h(n1) = 0, then there is some ksuch that {n0}(0) halts in exactly ksteps but {n1}(0) does not. But then there is no g∈H†with g(n0) = 1 and g(n1) = 1. Therefore H†does not shatter the set {n0, n1}. Now suppose that Ais a proper PAC learner for H, and let mbe a sample function for the induced PAC learner for H†. We will show that ∅′≤sW(A, m). LetM=m(ǫ, δ) for any choice of ǫ, δ∈(0,1). Given n∈N, letSn=/parenleftbig (n,1)/parenrightbig i∈[M], i.e., Mcopies of (n,1). Let zn=A(Sn) and let µnbe the measure with a single point mass on ( n,1). Note that µM n places full measure on Snand that for any map h:X → { 0,1}, its loss with respect to µnis either 0 or 1. Note that if {n}(0)↓then there is an n∗∈ I(namely, n∗=n) such that the minimum loss with respect to µis 0. AsAis a proper PAC learner for H, we must then have zn∼n. Otherwise, Aincurs an error of 1 > ǫwith probability 1 >(1−δ) over µnon samples of size M, producing contradiction with the PAC condition on m(ǫ, δ). On the other hand, if {n}(0)↑, then for any e∈ Iwe must have e/\e}atio\slash∼n, and in particular zn/\e}atio\slash∼n. Therefore n/mapsto→1−znis precisely the function ∅′. In particular, this shows that ∅′≤sW(A, m). /square 4.2.2. Rich index spaces. When the index space Iof a computable presentation of a hypothesis class is rich, we have lim I≡sWlimNN. In this case, Theorem 4.1shows that there is an ERM that is strongly Weihrauch reducible to lim NN. We also provide a partial converse in this situation, using t he notion of parallelization. We show that there is a computable presentation of a hypothesis clas s with rich index space such that lim NN is strongly Weihrauch reducible to the parallelization of a ny proper PAC learner for the presentation along with any sample function for the proper learner. Remark 4.7. It is worth taking a moment to discuss why, when considering l earners on continuum- sized metric spaces, we study the parallelization of the lea rner as opposed to the learner itself. When comparing the relative computational strength of two maps fandg, the notion of gbeing “more complex” than fcan be intuitively thought of as the statement that one can co mpute fwhen given access to g. This is made precise using the formalism of strong Weihrauc h reducibility, in which a single application of fmust be computed using a single application of g(possibly along with some uniform pre- and post-processing). However, the manner in which we are discussing learners, nam ely as maps from ( X × Y )<ω× X to {0,1}(as opposed to maps from ( X ×Y )<ωto{0,1}X), means that a single application of a learner can only return a single bit of information about its input. In co ntrast, lim NNis a map from NNtoNNfor which a single application contains countably many bits of i nformation. As such, our representation of learners is not well suited to be compared to lim NNif we (somewhat artificially) allow only a single application of the learner. We can overcome this obstacle by instead considering the parallelization of the learner, i.e., by allowing ourselves to simultaneously ask countably many questions of the learner, rather than a single one. This is what we do in Theorem 4.8(in the analogous setting for proper learners). By Lemma 2.16, any function is strongly Weihrauch reducible to its parall elization, and by Lemma 2.17, limNNis strongly Weihrauch equivalent to its own parallelizatio n. Hence not much is lost when estab- lishing that lim NNis a lower bound on the parallelization of a proper learner (a s opposed to a lower bound on the proper learner itself).ON COMPUTABLE LEARNING OF CONTINUOUS FEATURES 15 Theorem 4.8. There is a hypothesis class that is PAC learnable but which ad mits a computable presentation Hsuch that limNN≤sW(/hatwideA, m)whenever Ais a proper PAC learner for Handmis a sample function for the PAC learner for H†thatAinduces. Proof. LetXbe the product of computable metric spaces NandNN, and let the index space Ibe {(e, z)∈N×NN:{e}z(0)↓}with distance inherited from Xand ideal points of the form ( e, z) where zhas only finitely many nonzero values, ordered by when the res pective programs with oracles halt on input 0. Given ( e0, z0),(e1, z1)∈ X, we write ( e0, z0)∼(e1, z1) when (a) e0=e1and (b) {e0}z0(0)↓if and only if {e1}z1(0)↓, with program e0with oracle z0taking the same number of steps to halt on input 0 as does e1with oracle z1(when they both halt). Define H:I × X → { 0,1}by H/parenleftbig (e0, z0),(e1, z1)/parenrightbig =/braceleftBigg 1 (e0, z0)∼(e1, z1); 0 otherwise . Note that His computable because {e0}z0(0)↓for every ( e0, z0)∈ I. First we show that H†shatters no set of size 2, so that it has VC dimension 1 and henc e is PAC learnable by Theorem 2.23. Let ( e0, z0),(e1, z1)∈ X be distinct. If there exists an h∈H†with h(e0, z0) = 1 and h(e1, z1) = 0, then there is some ksuch that the program e0with oracle z0halts on input 0 in exactly ksteps but either e0/\e}atio\slash=e1or the program e1with oracle z1does not halt on input 0 in exactly ksteps. But then there is no g∈H†with g(e0, z0) = 1 and g(e1, z1) = 1. Therefore H† does not shatter the set {(e0, z0),(e1, z1)}. Now suppose that A: (X × Y )<ω→ I is a proper PAC learner for H, let Abe the induced PAC learner for H†, and let mbe a sample function for A(as a PAC learner for H†). We will show that J≤sW(/hatwideA, m). Then by Lemma 2.14, we will have lim NN≤sW(/hatwideA, m). Letz∈NN. We aim to uniformly compute z′usingA,m, and z. First we preprocess. Calculate k=m(ǫ, δ) for any choice of ǫ, δ∈(0,1) and construct the sequence Se,z=/parenleftbig ((e, z),1)k/parenrightbig e∈N. Then, apply /hatwideAto obtain a sequence ( ℓe, se)e∈N. Now consider the measure D(e,z)which places a pointmass on (( e, z),1). Because Ais a PAC learner, we have Pr S∼Dk (e,z)/parenleftBig/vextendsingle/vextendsingleLD(e,z)(A(S))−min w∈ILD(e,z)(/tildewideH(w))/vextendsingle/vextendsingle< ǫ/parenrightBig >1−δ. Therefore, as D(e,z)is a pointmass, we have/vextendsingle/vextendsingleLD(e,z)(A(Se,z))−min w∈ILD(e,z)(/tildewideH(w))/vextendsingle/vextendsingle< ǫ. Again because Ais a PAC learner and D(e,z)is atomic, we have an equivalence between the following statements: (1)A(Se,z)(e, z) = 1. (2)LD(e,z)(A(Se,z)) = 0. (3)LD(e,z)(/tildewideH(w)) = 0 for some w∈ I. (4)H(w,(e, z)) = 1 for some w∈ I. In particular, (3) ⇒(2) because Dk (e,z)concentrates mass on S(e,z), so otherwise Awould be guaranteed to incur a loss of 1 > ǫwhen trained on samples drawn from Dk (e,z), contradicting the PAC condition onm(ǫ, δ). Now note that if {e}z(0)↓, then there is a w= (e, z)∈ Isuch that H(w,(e, z)) = 1; by the previous equivalence, this implies that A(Se,z)(e, z) = 1. We are now equipped to post-process and calculate z′(n). If n/\e}atio\slash=ℓn, then A(Sn,z)(n, z) = 0 and, via¬(1)⇒ ¬(4) in the equivalence, {n}z(0)↑, meaning z′(n) = 0. Otherwise, n=ℓn. First compute {n}sn(0). This computation is guaranteed to halt, by definition ofIand the fact that Ais a proper learner. Let tbe the number of steps it took to halt. Next run {n}zon input 0 for tsteps. If it halts within tsteps, then {n}z(0)↓and so z′(n) = 1. If {n}zhas not halted on input 0 within tsteps, then A(Sn,z)(n, z) = 0 and the equivalence again implies that {n}z(0)↑, meaning z′(n) = 0. /squareON COMPUTABLE LEARNING OF CONTINUOUS FEATURES 16 Acknowledgements The authors would like to thank Caleb Miller for valuable dis cussion on the topic, particularly in helping refine the notion of computable PAC learning and in de scribing the computable algorithm for the decision stump. An extended abstract [ AAD+21] announcing related results in a different setting was prese nted at the Eighteenth International Conference on Computability and Complexity in Analysis (July 26–28, 2021). This material is based upon work supported by the National Sc ience Foundation under grant no. CCF-2106659. Freer’s work is funded in part by financial s upport from the Intel Probabilistic Computing Center. References [AAB+20] S. Agarwal, N. Ananthakrishnan, S. Ben-David, T. Lechne r, and R. Urner, On learnability with computable learners , Proceedings of the 31st International Conference on Algor ithmic Learning Theory (ALT) (San Diego, California, USA), Proceedings of Machine Learning R esearch, vol. 117, 2020, pp. 48–60. [AAD+21] N. Ackerman, J. Asilis, J. Di, C. Freer, and J.-B. Tristan ,On the computable learning of continuous features , Eighteenth International Conference on Computability and Complexity in Analysis, 2021. [BEHW89] A. Blumer, A. Ehrenfeucht, D. Haussler, and M. K. Wa rmuth, Learnability and the Vapnik–Chervonenkis dimension , J. ACM 36(1989), no. 4, 929–965. [Ber14] A. A. Beros, Learning theory in the arithmetic hierarchy , J. Symb. Log. 79(2014), no. 3, 908–927. [BGP21] V. Brattka, G. Gherardi, and A. Pauly, Weihrauch complexity in computable analysis , Handbook of Com- putability and Complexity in Analysis (V. Brattka and P. Her tling, eds.), Springer, Cham, 2021, pp. 367– 417. [BHW08] V. Brattka, P. Hertling, and K. Weihrauch, A tutorial on computable analysis , New Computational Paradigms, Springer, 2008, pp. 425–491. [BP03] V. Brattka and G. Presser, Computability on subsets of metric spaces , Theoret. Comput. Sci. 305 (2003), no. 1-3, 43–76. [Cal15] W. Calvert, PAC learning, VC dimension, and the arithmetic hierarchy , Arch. Math. Logic 54(2015), no. 7-8, 871–883. [CMPR21] T. Crook, J. Morgan, A. Pauly, and M. Roggenbach, A computability perspective on (verified) machine learning , arXiv e-print 2102.06585 (2021). [FW95] S. Floyd and M. Warmuth, Sample compression, learnability, and the Vapnik–Chervon enkis dimension , Machine Learning 21(1995), no. 3, 269–304. [Gol67] E. M. Gold, Language identification in the limit , Inform. and Control 10(1967), no. 5, 447–474. [HR09] M. Hoyrup and C. Rojas, Computability of probability measures and Martin-L¨ of ran domness over metric spaces , Inform. and Comput. 207 (2009), no. 7, 830–847. [SB14] S. Shalev-Shwartz and S. Ben-David, Understanding machine learning: From theory to algorithms , Cam- bridge University Press, 2014. [Sol08] D. Soloveichik, Statistical learning of arbitrary computable classifiers , arXiv e-print 0806.3537 (2008). [Val84] L. G. Valiant, A theory of the learnable , Commun. ACM 27(1984), no. 11, 1134–1142. [VC71] V. N. Vapnik and A. Y. Chervonenkis, On the uniform convergence of relative frequencies of event s to their probabilities , Theor. Probability Appl. 16(1971), no. 2, 264–280. [Wei00] K. Weihrauch, Computable analysis: An introduction , Springer, 2000. Harvard University, Cambridge, MA 02138, USA Email address :nate@aleph0.net Computer Science Department, Boston College, Chestnut Hil l, MA 02467, USA Email address :julian.asilis@bc.edu Department of Mathematics, Boston College, Chestnut Hill, MA 02467, USA Email address :dij@bc.edu Department of Brain and Cognitive Sciences, Massachusetts I nstitute of Technology, Cambridge, MA 02139, USA Email address :freer@mit.edu Computer Science Department, Boston College, Chestnut Hil l, MA 02467, USA Email address :tristanj@bc.edu

You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you s... You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you select really corresponds to the excerpt: there will be details mentioned in the excerpt that should appear in the paper. If you read an abstract and it seems like it could be the paper we’re looking for, read the paper to make sure. Also: sometimes you’ll read a paper that cites the paper we’re looking for. In such cases, please go to the references in order to find the full name of the paper we’re looking for, and search for it, and then select it. To the best of our knowledge, there is only one method, QII, that provably provides differentially private black-box post-hoc model explanations [CITATION], protecting the explanation data. QII introduces Shapley value based model explanations, which have become a popular model explanation framework You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you select really corresponds to the excerpt: there will be details mentioned in the excerpt that should appear in the paper. If you read an abstract and it seems like it could be the paper we’re looking for, read the paper to make sure. Also: sometimes you’ll read a paper that cites the paper we’re looking for. In such cases, please go to the references in order to find the full name of the paper we’re looking for, and search for it, and then select it. To the best of our knowledge, there is only one method, QII, that provably provides differentially private black-box post-hoc model explanations [CITATION], protecting the explanation data. QII introduces Shapley value based model explanations, which have become a popular model explanation framework

- Paper ID: b2681fbc27089797cedc1a454b0c1b94c6482abe Title: Algorithmic Transparency via Quantitative Input Influence: Theory and Experim... - Paper ID: b2681fbc27089797cedc1a454b0c1b94c6482abe Title: Algorithmic Transparency via Quantitative Input Influence: Theory and Experiments with Learning Systems Abstract: Algorithmic systems that employ machine learning play an increasing role in making substantive decisions in modern society, ranging from online personalization to insurance and credit decisions to predictive policing. But their decision-making processes are often opaque-it is difficult to explain why a certain decision was made. We develop a formal foundation to improve the transparency of such decision-making systems. Specifically, we introduce a family of Quantitative Input Influence (QII) measures that capture the degree of influence of inputs on outputs of systems. These measures provide a foundation for the design of transparency reports that accompany system decisions (e.g., explaining a specific credit decision) and for testing tools useful for internal and external oversight (e.g., to detect algorithm... - Paper ID: b2681fbc27089797cedc1a454b0c1b94c6482abe Title: Algorithmic Transparency via Quantitative Input Influence: Theory and Experiments with Learning Systems Abstract: Algorithmic systems that employ machine learning play an increasing role in making substantive decisions in modern society, ranging from online personalization to insurance and credit decisions to predictive policing. But their decision-making processes are often opaque-it is difficult to explain why a certain decision was made. We develop a formal foundation to improve the transparency of such decision-making systems. Specifically, we introduce a family of Quantitative Input Influence (QII) measures that capture the degree of influence of inputs on outputs of systems. These measures provide a foundation for the design of transparency reports that accompany system decisions (e.g., explaining a specific credit decision) and for testing tools useful for internal and external oversight (e.g., to detect algorithmic discrimination). Distinctively, our causal QII measures carefully account for correlated inputs while measuring influence. They support a general class of transparency queries and can, in particular, explain decisions about individuals (e.g., a loan decision) and groups (e.g., disparate impact based on gender). Finally, since single inputs may not always have high influence, the QII measures also quantify the joint influence of a set of inputs (e.g., age and income) on outcomes (e.g. loan decisions) and the marginal influence of individual inputs within such a set (e.g., income). Since a single input may be part of multiple influential sets, the average marginal influence of the input is computed using principled aggregation measures, such as the Shapley value, previously applied to measure influence in voting. Further, since transparency reports could compromise privacy, we explore the transparency-privacy tradeoff and prove that a number of useful transparency reports can be made differentially private with very little addition of noise. Our empirical validation with standard machine learning algorithms demonstrates that QII measures are a useful transparency mechanism when black box access to the learning system is available. In particular, they provide better explanations than standard associative measures for a host of scenarios that we consider. Further, we show that in the situations we consider, QII is efficiently approximable and can be made differentially private while preserving accuracy. Citation Count: 666

You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you s... You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you select really corresponds to the excerpt: there will be details mentioned in the excerpt that should appear in the paper. If you read an abstract and it seems like it could be the paper we’re looking for, read the paper to make sure. Also: sometimes you’ll read a paper that cites the paper we’re looking for. In such cases, please go to the references in order to find the full name of the paper we’re looking for, and search for it, and then select it. With the success of various attacking methods, robustness becomes an important consideration in watermarking techniques. However, [CITATION] proves that it is only feasible to achieve robustness to a well-specified set of attacks, instead of all. This fact aligns with our Theorem 4.4, which characterizes the fundamental limits of robust watermarking under different attacking powers. You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you select really corresponds to the excerpt: there will be details mentioned in the excerpt that should appear in the paper. If you read an abstract and it seems like it could be the paper we’re looking for, read the paper to make sure. Also: sometimes you’ll read a paper that cites the paper we’re looking for. In such cases, please go to the references in order to find the full name of the paper we’re looking for, and search for it, and then select it. With the success of various attacking methods, robustness becomes an important consideration in watermarking techniques. However, [CITATION] proves that it is only feasible to achieve robustness to a well-specified set of attacks, instead of all. This fact aligns with our Theorem 4.4, which characterizes the fundamental limits of robust watermarking under different attacking powers.

No papers were found for the given search query. Please use a different query. No papers were found for the given search query. Please use a different query. No papers were found for the given search query. Please use a different query.

- Paper ID: d9e1c06bcac34fa735152a42c2d903ab67435104 Title: Digital Image Watermarking Techniques: A Review Abstract: Digital image au... - Paper ID: d9e1c06bcac34fa735152a42c2d903ab67435104 Title: Digital Image Watermarking Techniques: A Review Abstract: Digital image authentication is an extremely significant concern for the digital revolution, as it is easy to tamper with any image. In the last few decades, it has been an urgent concern for researchers to ensure the authenticity of digital images. Based on the desired applications, several suitable watermarking techniques have been developed to mitigate this concern. However, it is tough to achieve a watermarking system that is simultaneously robust and secure. This paper gives details of standard watermarking system frameworks and lists some standard requirements that are used in designing watermarking techniques for several distinct applications. The current trends of digital image watermarking techniques are also reviewed in order to find the state-of-the-art methods and their limitations. Some conventional attacks are discussed, and future research direction... - Paper ID: d9e1c06bcac34fa735152a42c2d903ab67435104 Title: Digital Image Watermarking Techniques: A Review Abstract: Digital image authentication is an extremely significant concern for the digital revolution, as it is easy to tamper with any image. In the last few decades, it has been an urgent concern for researchers to ensure the authenticity of digital images. Based on the desired applications, several suitable watermarking techniques have been developed to mitigate this concern. However, it is tough to achieve a watermarking system that is simultaneously robust and secure. This paper gives details of standard watermarking system frameworks and lists some standard requirements that are used in designing watermarking techniques for several distinct applications. The current trends of digital image watermarking techniques are also reviewed in order to find the state-of-the-art methods and their limitations. Some conventional attacks are discussed, and future research directions are given. Citation Count: 82 - Paper ID: 1275d063dfae9ea716eaad5b027555501ff4f2a0 Title: Analysis of Digital Image Watermarking Techniques through Hybrid Methods Abstract: Digital image watermarking is an attractive research area since it protects the multimedia data from unauthorized access. For designing an efficient and robust digital image watermarking system, the trade-off among imperceptibility, robustness, capacity, and security must be maintained. Various studies regarding this concern have been performed to ensure these requirements by hybridizing different domains, such as spatial and transform domains. In this paper, we have presented an analytical study of the existing hybrid digital image watermarking methods. At first, we have given a standard framework for designing a hybrid method that ensures the basic design requirements of watermarking for various applications. After a brief literature review, we compared and analyzed the complexity of several existing hybrid methods in a tabular form. The limitations and applications of these methods are also highlighted. Finally, we summarized the challenges of the existing methods and concluded the study by giving future research directions. Citation Count: 24 - Paper ID: 9fb530384dd7fbe29d041d85cdf1b3196ca09bb0 Title: State-of-the-Art in Techniques of Text Digital Watermarking: Challenges and Limitations Abstract: Data protection from malicious attacks and misuse has become a crucial issue. Various types of data, including images, videos, audio and text documents, have given cause for the development of different methods for their protection. Cryptography, digital signatures and steganography are the most well known technologies used to protect data. During the last decade, digital watermarking technology has also been utilized as an alternative to prevent media forgery and tampering or falsification to ensure both copyright and authentication. Much work has been done to protect images, videos and audio but only a few algorithms have been considered for text document protection with digital watermarking. However, our survey observed that available text watermarking algorithms are neither robust nor imperceptible and as such remain unsecured methods of protection. Hence, research to improve the performance of text watermarking algorithms is required. This paper reviews current watermarking algorithms for text documents and categorizes text watermarking methods based on the way the text was treated during the watermarking process. It also discusses merits and demerits of available methods as well as recent proposed methods for evaluating text watermarking systems and the need for further research on digital text watermarking. Citation Count: 17 - Paper ID: 2a50ef6691202dbf1ac2dec778f7bfb33c069eb5 Title: A Robust Watermarking Scheme against Frame Blending, Projection and Content Adaptation Attacks Abstract: Camcorder based video copy attack has become a serious threat to the electronic movie distribution services. In literature, it is found that main distortions caused during cam-cording are frame blending and projection attacks. Though, several watermarking schemes exist to handle geometric distortions caused during cam-cording, very few of them are robust against temporal distortion. Based on the existing limitations of the state-of-art literature, a blind watermarking scheme which is robust against both frame blending and projection attacks is proposed in this paper. Using scale invariant feature transform (SIFT) and higher-order singular value decomposition (HOSVD), the temporal synchronization between cam-corded video and source video is achieved. Further, SIFT and log-polar transform are used together to achieve spatial synchronization of the embedding locations. The main contribution of this work is twofold, primarily it handles the frame blending and projection attacks during cam-cording and secondly it prevents the distortion in the watermark signal when different content adaptation is done specially when the quality and temporal adaptation is done in case of scalable video. A comprehensive set of experiments has been carried out to show the applicability of the proposed scheme over the existing literature. Citation Count: 1 - Paper ID: 2eb0dfddc2c1447b9fdee9b47bc8621b67e77161 Title: A robust software watermarking framework using shellcode Abstract: None Citation Count: 1 - Paper ID: 31e1bc764f2e55021749b340e2237c7e88c2cc04 Title: Software and Hardware FPGA-Based Digital Watermarking and Steganography Approaches: Toward New Methodology for Evaluation and Benchmarking Using Multi-Criteria Decision-Making Techniques Abstract: Evaluating and benchmarking software and hardware field programmable gate array (FPGA)-based digital watermarking are considered challenging tasks because of multiple and conflicting evaluation criteria. A few evaluation and benchmarking techniques/frameworks have been implemented to digital watermarking or steganography; however, these approaches still present certain limitations. In particular, fixing some attributes on account of other attributes and well-known benchmarking approaches are limited to robust watermarking techniques. Thus, this study aims toward a new methodology for evaluation and benchmarking using multi-criteria analysis for software and hardware “FPGA”-based digital watermarking or steganography. To achieve this objective, two iterations are conducted. The first iteration consists of two stages: discussing software and hardware “FPGA”-based digital watermarking or steganography to create a dataset with various samples for benchmarking and discussing the evaluation method and then disc... Citation Count: 54 - Paper ID: 694e62aaa1d41d1878d53f05106a2370ee9d1411 Title: Implementation of Robust Digital Watermarking Algorithms using SVD and DCT Techniques Abstract: Copyright risks in multimedia are increasing as a result of large numbers of computers in network. A technique used to preserve rights is watermark implementation to authenticate authorship. This information which is embedded in the original image is a “Digital Watermark”, which might be visible or invisible. For this technique to become more efficient, the Watermark should be robust so that its removal from the original image by the hackers becomes difficult. In time, various Watermarking algorithms are developed. However, each of them has its own advantages and limitations. Recent research shows that SVD (Singular Value Decomposition) algorithms are used because of their simple scheme and mathematical function. Another algorithm is DCT (Discrete Cosine Transform). Each of such algorithms has its own advantages, a combination of their advantages is the RST (Rotation, Scaling, Translation) algorithm. This paper shall attempt to compare algorithms and provide a conclusion on their performance and finally implementation in Hardware, in Xilinx Virtex II Pro Citation Count: 1 - Paper ID: 8b79be483a7668bd75d0f0a378eed8cffbe53c82 Title: A Proposed Robust Video Watermarking Algorithm: Enhanced Extraction from Geometric Attacks Abstract: This paper proposes a new integrated system based on a video watermarking algorithm with high imperceptibility, improved security, robust against common image and video processing attacks and geometric attacks. The proposed algorithm is based on the features of a selected best algorithm by Agilandeeswari and Ganesan and its limitations. The advantages of the hybrid transform techniques of Nonsubsampled Contourlet Transform, Discrete Wavelet Transform and Singular Value Decomposition are utilized for high robustness against common attacks. The features of angle invariance and distance invariability of Log Polar Transform and Inverse Log Polar Transform are used in the extraction process only to resist geometric attacks without affecting the imperceptibility of watermark and watermarked video. Modified Arnold Transformation is introduced to improve the security of the watermark. The experiment shows that the algorithm is extremely robust in terms of most common attacks and geometric attacks with high imperceptibility and improved security. Citation Count: 5 - Paper ID: 3df70b202aaf21d2f0b9f647c7aea68e586f4e75 Title: A Somewhat Robust Image Watermark against Diffusion-based Editing Models Abstract: Recently, diffusion models (DMs) have become the state-of-the-art method for image synthesis. Editing models based on DMs, known for their high fidelity and precision, have inadvertently introduced new challenges related to image copyright infringement and malicious editing. Our work is the first to formalize and address this issue. After assessing and attempting to enhance traditional image watermarking techniques, we recognize their limitations in this emerging context. In response, we develop a novel technique, RIW (Robust Invisible Watermarking), to embed invisible watermarks leveraging adversarial example techniques. Our technique ensures a high extraction accuracy of $96\%$ for the invisible watermark after editing, compared to the $0\%$ offered by conventional methods. We provide access to our code at https://github.com/BennyTMT/RIW. Citation Count: 0 - Paper ID: c864923e6ee22d87a4b8b32584c4923c537767be Title: Spread-spectrum audio watermarking: requirements, applications, and limitations Abstract: Watermarking has been adopted as a technology of choice for many applications related to e-commerce of audio content. We present a brief summary of a set of spread-spectrum watermarking techniques for effective covert communication over an audio signal carrier. Watermark robustness is enabled using redundant spread-spectrum for prevention against de-synchronization attacks. We improve watermark inaudibility by detecting and not watermarking blocks of audio where a spread spectrum sequence, if added to the frequency spectrum, would be audible. Finally, we overview the security limitations of our technology with respect to parameter selection and position it with respect to three main applications of watermarking: (a) content screening, (b) tracing unlicensed content distribution, and (c) robust metadata. Citation Count: 47

information Review Digital Image Watermarking Techniques: A Review Mahbuba Begum1,* and Mohammad Shorif Uddin2 1Department of Computer Sci... information Review Digital Image Watermarking Techniques: A Review Mahbuba Begum1,* and Mohammad Shorif Uddin2 1Department of Computer Science and Engineering, Mawlana Bhashani Science and Technology University, Tangail-1902, Bangladesh 2Department of Computer Science and Engineering, Jahangirnagar University, Savar, Dhaka-1342, Bangladesh; shorifuddin@gmail.com *Correspondence: mahbuba327@yahoo.com Received: 11 January 2020; Accepted: 13 February 2020; Published: 17 February 2020 /gid00030/gid00035/gid00032/gid00030/gid00038/gid00001/gid00033/gid00042/gid00045 /gid00001 /gid00048/gid00043/gid00031/gid00028/gid00047/gid00032/gid00046 Abstract: Digital image authentication is an extremely significant concern for the digital revolution, as it is easy to tamper with any image. In the last few decades, it has been an urgent concern for researchers to ensure the authenticity of digital images. Based on the desired applications, several suitable watermarking techniques have been developed t... information Review Digital Image Watermarking Techniques: A Review Mahbuba Begum1,* and Mohammad Shorif Uddin2 1Department of Computer Science and Engineering, Mawlana Bhashani Science and Technology University, Tangail-1902, Bangladesh 2Department of Computer Science and Engineering, Jahangirnagar University, Savar, Dhaka-1342, Bangladesh; shorifuddin@gmail.com *Correspondence: mahbuba327@yahoo.com Received: 11 January 2020; Accepted: 13 February 2020; Published: 17 February 2020 /gid00030/gid00035/gid00032/gid00030/gid00038/gid00001/gid00033/gid00042/gid00045 /gid00001 /gid00048/gid00043/gid00031/gid00028/gid00047/gid00032/gid00046 Abstract: Digital image authentication is an extremely significant concern for the digital revolution, as it is easy to tamper with any image. In the last few decades, it has been an urgent concern for researchers to ensure the authenticity of digital images. Based on the desired applications, several suitable watermarking techniques have been developed to mitigate this concern. However, it is tough to achieve a watermarking system that is simultaneously robust and secure. This paper gives details of standard watermarking system frameworks and lists some standard requirements that are used in designing watermarking techniques for several distinct applications. The current trends of digital image watermarking techniques are also reviewed in order to find the state-of-the-art methods and their limitations. Some conventional attacks are discussed, and future research directions are given. Keywords: LSB; DCT; DFT; DWT; SVD 1. Introduction Image processing and the internet have made it easier to duplicate, modify, reproduce, and distribute digital images at low cost and with approximately immediate delivery without any degradation of quality. Network technology has been developing and progressing so quickly that it threatens the privacy and security of data. Therefore, content authentication, copyright protection, and protection against duplication play an essential role in facing the challenges of the existing and upcoming threats in maintaining digital information. Digital image watermarking is simply the digital watermarking of an image, which provides an alternative solution for ensuring tamper-resistance, the ownership of intellectual property, and reinforcing the security of multimedia documents. Any digital content, such as images, audio, and videos, can hide data. Digital content can easily be illegally possessed, duplicated, and distributed through a physical transmission medium during communications, information processing, and data storage. Digital image watermarking is a technique in which watermark data is embedded into a multimedia product and, later, is extracted from or detected in the watermarked product. These methods ensure tamper-resistance, authentication, content verification, and integration of the image [ 1]. It is not very easy to eliminate a watermark by displaying or converting the watermarked data into other file formats. Therefore, after an attack, it is possible to obtain information about the transformation from the watermark. To discern the di erence between digital watermarking and other technologies such as encryption is essential [ 2]. Digital-to-analog conversion, compression, file format changes, re-encryption, and decryption can also be survived through digital image watermarking techniques. These tasks make it an alternative (or complementary) to cryptography. The information is embedded in the content and cannot be removed by normal usage [3]. The word “steganography” is derived from the Greek word “steganos.” This technique conceals communication and changes an image such that only the sender and the intended receiver can identify Information 2020 ,11, 110; doi:10.3390 /info11020110 www.mdpi.com /journal /informationInformation 2020 ,11, 110 2 of 38 the sent message. This technique makes detection a more di cult task. Instead of encrypting messages, steganography can be used to hide them in other ino ensive-looking objects, so their existence is not discovered and, therefore, can be used as an alternative tool for privacy and security. However, due to the rapid proliferation of internet and computer networks, steganography can be used as a tool for exchanging information and planning terrorist attacks [ 3]. Steganography hides the existence of a cover image, while a watermarking technique embeds a message into the actual content of the digital signal within the signal itself. Therefore, an eavesdropper cannot remove or replace a message to obtain an output message. To protect content from unauthorized access, embedding information into the original image is essential. Digital image watermarking is imperceptible and hard to remove by unauthorized persons. The technique has been implemented by various algorithms using the spatial and frequency domains, each having their distinct benefits and boundaries. The contributions of this research are as follows:  We identify the limitations of existing watermarking techniques;  We present the current trends of image watermarking techniques;  We investigate the techniques that meet some of the requirements of image watermarking techniques perfectly;  We point out the challenges that must be addressed by future researchers. In this paper, the framework of general watermarking methods, separated into the processes of embedding and extracting watermarks, along with a general background, is shortly revised in the first section. Some standard design requirements for evaluating the performance of watermarking systems are listed in the following subsections. Related applications that make watermarking systems a highly focused research area are also described. Based on the working domain, a survey of digital image watermarking techniques is subsequently presented. Then, a summary of the research results of the discussed state-of-the-art methods and current trends in the field is described in tabular format. Next, we list some conventional attacks or threats which must be treated as a challenge for designing an ecient system. Performance metrics, such as peak-signal-to-noise ratio (PSNR), structural similarity index (SSIM), mean squared error (MSE), and normalized cross-correlation (NCC), are also briefly described. Finally, the last section concludes the study. 2. Image Watermarking Backgrounds and Frameworks At present, digital content can be spread easily through communication channels due to the rapid rise of global computer networks, the internet, and multimedia systems. To protect digital information against illegal possession, duplication, manipulation, usage, and distribution through physical transmission media during communications, information processing, and data storage, digital image watermarking makes it possible to construct a platform for researchers by considering it as a research area. Paper watermarks began as early in 1282 and, therefore, digital watermarking techniques have been improved by integrating paper configuration, quality, and quantity considerations. Watermarking has been used broadly for enhancing security [ 4]. The computerized technology of digital watermarking appeared in 1988, providing confidentiality, integrity, and availability, and various innovations regarding digital image watermarking have been incorporated since 1995. In watermarking techniques, a symbol of owner authenticity (watermark) is embedded into the host signal and, later, this watermark data can be extracted. The watermark data, which may be visible or invisible, can contain a single bit, a set of binary data, or a number of samples in the host signal [5]. To imitate the human visual perception system, information entropy plays an essential role in the digital image watermarking scheme. To achieve an optimal balance between imperceptibility, robustness, and capacity of a digital image watermarking technology, information entropy can be used through a Just Noticeable Di erence (JND) model [ 6]. Information entropy can be defined in terms of masking e ect and can be utilized to determine the positions at which the data are inserted. ThisInformation 2020 ,11, 110 3 of 38 scenario minimizes perceptual distortion and gives better robustness and good imperceptibility. The entropy of a system with n states can be defined by the following equations [7]: Information Entropy, ETP=nX i=1PilogP i (1) where 0Pi1 and nX i=1Pi=1 (2) where Pidenotes the probability of occurrence for the event i. For the secured communication of a message, the process begins with a cover image (host image). The host image can be considered purely as noise, noise with side information, or as a multimedia message that must be transmitted. The watermarked data passes through a communication channel, which may be lossy, noisy, or unreliable. Hence, the watermarked data may su er from possible attacks, including lossy compression, geometric distortion, signal processing operations, and signal conversion, among others; that is, there may be a di erence between the original watermarked data and the received data [ 8]. A watermarked image passes through a communication channel, which incurs noise. This noise maximizes the information entropy [ 9], which increases the uncertainty or ambiguity of the average information contained in an image. Watermarking techniques can, thus, only be applied to high-resolution and complex-patterned images that have higher information entropy. Therefore, to improve security, the encoded image must be processed such that the image reconstruction will be robust. A new optical image encoding method obtains the encoded image by a random-phase encoding technique in both the input and the Fourier planes [ 10], in which the two random phase plates at the input and the Fourier planes are replaced by two deformable mirrors, respectively. Thus, the system can achieve arbitrary beam shaping in the amplitude and phase information of the image [11]. For a secure communication model, the digital image watermarking process consists of a watermark embedding part and a watermark extraction part. In the watermark embedding part, at first, the cover image is pre-processed, and then, its entropy is evaluated to find the integrating capacity information of the image. Then, using an optical image encoding method, the encoder embeds a watermark image into the high entropy value of the host image by using a secret key. Then, the system achieves the amplitude and phase shaping information of a laser beam and generates the watermarked image. The watermark embedding part is depicted in Figure 1a. Finally, in the watermark extraction step, the watermarked image is pre-processed. After that, the system extracts the amplitude and phase shaping information of the laser beam patterns. Then, the entropy of these beam patterns is evaluated. A high entropy value is selected for extracting the watermark, in order to ensure better robustness and imperceptibility. A decoder detects the watermark image as output from the watermarked image using the same key, as depicted in Figure 1b. The system demonstrates that image reconstruction of the watermark image from the watermarked image is simple, robust, and imperceptible. The watermark embedding process generates a watermarked image, DW, which can be described by the following function: Watermarked Image ,DW=E(I,ETP,W,K), (3) where Eis the encoding algorithm, Iis the cover image, ETP is the information entropy, Wis the watermark image, and Kis the security key. The watermark extraction process extracts the watermark image, W’, which can be described by the following decoder function, where e(.)is the decoding algorithm: W0=e(DW,K,ETP,I). (4)Information 2020 ,11, 110 4 of 38 Information 2020 , 11, x FOR PEER REVIEW 4 of 42 (a) (b) Figure 1. (a) Watermark embedding and ( b) watermark extraction. The watermark embedding process generates a watermarked image, DW, which can be described by the following function: 𝑊𝑎𝑡𝑒𝑟𝑚𝑎𝑟𝑘𝑒𝑑 𝐼𝑚𝑎𝑔𝑒, 𝐷 ௐ=𝐸 (𝐼, 𝐸𝑇𝑃, 𝑊, 𝐾 ), (3) where E is the encoding algorithm, I is the cover image, ETP is the information entropy, W is the watermark image, and K is the security key. The watermark extraction process extracts the watermark image, W’, which can be described by the following decoder function, where e(.) is the decoding algorithm: 𝑊ᇱ=𝑒 ൫ 𝐷 ௐ, 𝐾 ,𝐸 𝑇 𝑃 ,𝐼 ൯ . (4) 3. Design Requirements of Image Watermarking System Digital image watermarking techniques add a watermark into multimedia data to ensure authenticity and to protecting a copyright holder from the unauthorized mani pulation of their data [12]. Hence, it is necessary to define the requirements or characteristics of a watermarking system, which are listed in the following subsections. Figu re 2 illustrates the requirements of watermarking techniques. Based on applications, these requirem ents evaluate the performance of watermarking systems. Figure 1. (a) Watermark embedding and ( b) watermark extraction. 3. Design Requirements of Image Watermarking System Digital image watermarking techniques add a watermark into multimedia data to ensure authenticity and to protecting a copyright holder from the unauthorized manipulation of their data [ 12]. Hence, it is necessary to define the requirements or characteristics of a watermarking system, which are listed in the following subsections. Figure 2 illustrates the requirements of watermarking techniques. Based on applications, these requirements evaluate the performance of watermarking systems. Information 2020 , 11, x FOR PEER REVIEW 5 of 42 Figure 2. Design requirements for an image watermarking system. 3.1. Imperceptibility Imperceptibility is key in evaluating the performance of a watermarking system. It is represented by invisibility and fidelity. In this ca se, the watermarked image must appear the same as the original image. They should be perceptua lly indistinguishable to humans, despite a minor degradation in brightness or image contrast. Thus, the image quality must not be affected . There are different methods for evaluating the imperceptibility of a watermarking system. Yang et al. [13] have proposed a new method based on the masking features of the human visual system. Their proposed method, Masking-based Peak Signal to Noise Ratio (MPSNR), performed better in evaluating the imperceptibility of watermarking systems. Their experimental results showed that higher masking strength provides less quality degradation in the watermarked image. For improving watermark imperceptibility, larger singular values are sele cted for inserting the watermark image, based on watermark capacity [14]. In this regard, the structural similarity index (SSIM) is used to evaluate watermark imperceptibility. The quality of the watermarked image may be lost due to the watermark embedding process. However, in invisible watermarki ng, it is often unnoticeable by the human visual system. High peak-signal-to-noise ratio (PSNR) results imply better imperceptibility. The best watermarking techniques ensure better imperceptibility, meaning that they generate no visual difference between the host image and the waterm arked image. One method [15] selects the best region of the cover image for watermark insertion to achieve better imperceptibility. A visible watermark in an image is sometimes preferred [1 6]. However, invisible wa termarking systems are more popular. Imperceptibility may be used in di gital imaging, telemedicine , digital documents, and so on. 3.2. Robustness Robustness is the requirement that a watermark is able to be detected after some common signal processing manipulation operations in digital imag e watermarking systems have been applied. These operations include spatial filtering, color ma pping, scanning and printing, lossy compression, scaling, translation, and rotation . They also include other operations, such as analog to digital (A/D) conversions, digital to analog (D/A) conversions, image enhancement, cutting, and so on [2]. There exist several general approaches to achieving high robustness, such as redundant embedding, spread spectrum, and embedding watermarks, among others. Therefore, a good digital image watermarking system should be robust against various attacks, such that unauthorized distributors cannot remove or exclude watermark data. Depending on the application, not all watermarking algorithms may have robustness at the same level. Some are robust against different image processing operations, while some fail against other attacks [17]. Therefore, robustness can be classified into robust, fragile, and semi-fragile. Figure 2. Design requirements for an image watermarking system.Information 2020 ,11, 110 5 of 38 3.1. Imperceptibility Imperceptibility is key in evaluating the performance of a watermarking system. It is represented by invisibility and fidelity. In this case, the watermarked image must appear the same as the original image. They should be perceptually indistinguishable to humans, despite a minor degradation in brightness or image contrast. Thus, the image quality must not be a ected. There are di erent methods for evaluating the imperceptibility of a watermarking system. Yang et al. [ 13] have proposed a new method based on the masking features of the human visual system. Their proposed method, Masking-based Peak Signal to Noise Ratio (MPSNR), performed better in evaluating the imperceptibility of watermarking systems. Their experimental results showed that higher masking strength provides less quality degradation in the watermarked image. For improving watermark imperceptibility, larger singular values are selected for inserting the watermark image, based on watermark capacity [ 14]. In this regard, the structural similarity index (SSIM) is used to evaluate watermark imperceptibility. The quality of the watermarked image may be lost due to the watermark embedding process. However, in invisible watermarking, it is often unnoticeable by the human visual system. High peak-signal-to-noise ratio (PSNR) results imply better imperceptibility. The best watermarking techniques ensure better imperceptibility, meaning that they generate no visual di erence between the host image and the watermarked image. One method [ 15] selects the best region of the cover image for watermark insertion to achieve better imperceptibility. A visible watermark in an image is sometimes preferred [ 16]. However, invisible watermarking systems are more popular. Imperceptibility may be used in digital imaging, telemedicine, digital documents, and so on. 3.2. Robustness Robustness is the requirement that a watermark is able to be detected after some common signal processing manipulation operations in digital image watermarking systems have been applied. These operations include spatial filtering, color mapping, scanning and printing, lossy compression, scaling, translation, and rotation. They also include other operations, such as analog to digital (A /D) conversions, digital to analog (D /A) conversions, image enhancement, cutting, and so on [ 2]. There exist several general approaches to achieving high robustness, such as redundant embedding, spread spectrum, and embedding watermarks, among others. Therefore, a good digital image watermarking system should be robust against various attacks, such that unauthorized distributors cannot remove or exclude watermark data. Depending on the application, not all watermarking algorithms may have robustness at the same level. Some are robust against di erent image processing operations, while some fail against other attacks [ 17]. Therefore, robustness can be classified into robust, fragile, and semi-fragile.  Robust: A robust watermark prevents various noisy attacks, as well as geometric or non-geometric attacks, without altering the watermark data. The watermark remains the same even after some attacks and provides authorization by detecting the watermark [ 18]. This watermark is used in such areas as copyright protection, broadcast monitoring, copy control, and fingerprinting [16].  Fragile: Fragile watermarks are mainly used for integrity verification and content authentication of multimedia data where signature information can be added. This watermark validates whether it has been tampered or not [ 16]. A fragile technique is typically easier to implement than a robust one [ 19]. In [ 20], binary authentication information was inserted into the host image where, for identifying tampering and localization, a pixel-based fragile watermarking technique was used. It resulted in an acceptable visual e ect (in terms of the human eye).  Semi-fragile: This type of watermark resists some transformations but fails after malicious transformations. A semi-fragile watermark can be used for image authentication [21]. An all phase bi-orthogonal transform (APBT) and Singular Value Decomposition (SVD)-based algorithm has been proposed [ 22] to achieve better robustness and imperceptibility of the watermarkingInformation 2020 ,11, 110 6 of 38 system, where the block-based APBT algorithm is applied in a certain neighborhood obtained by selected candidate feature points. The coe cients of APBT generate the coe cient matrix for SVD to embed the watermark. Furthermore, a Discrete Wavelet Transform (DWT), all phase discrete cosine biorthogonal transform (APDCBT), and SVD-based method has been proposed [ 23] to enhance the imperceptibility and robustness, where the direct current (DC) coe cients of high-frequency sub-bands (LH and HL) are used to insert a watermark image. This method has been shown to be robust against many signal processing operations. 3.3. Security Watermarking algorithms that are not secure cannot be applied in copyright protection, data authentication, fingerprinting, and tracking of digital content. Therefore, security is a significant concern in digital image watermarking techniques. Security can be confirmed by various encryption methods, where the key decides the degree of security. Several methods, such as chaos-based, Discrete Cosine Transform (DCT), and logistic map-based encryption techniques, have been used to ensure the security and confidentiality of the embedded watermark [ 24]. The security of functional magnetic resonance imaging (fMRI) images is important, as they are related to brain activities. A watermarking method has been proposed for ensuring the integrity and authenticity of fMRI images [ 25], where a fragile reversible watermarking scheme was introduced to characterize fMRI images which are free from any format. The scheme is not dependent on using external metadata. In [ 26], binary pseudo-random sequences were used to encrypt the watermark before embedding, enhancing the security of the watermarking algorithm. The security requirement can be applied in telemedicine, digital imaging, telecommunications, multimedia data, etc. 3.4. Capacity Watermarking capacity (also known as payload) evaluates how much information can be inserted into the host image, based on the size of the original data. The capacity is defined by the number of bits carried by each host image after inserting the watermark image. However, it is a di cult task to insert more watermark information, which needs a pre-requisite based on practical applications [ 1]. In other words, the capacity determines the limitations of the watermarking information while satisfying watermarking robustness and imperceptibility. The available information to attackers, data encoder and decoder, distortion constraints, and the statistical model used in the cover image determine the watermarking capacity [ 27]. Various methods exist for evaluating watermarking capacity problems under attacks. These include game-theoretic and parallel Gaussian channels (PGC) approaches. On the other hand, watermark extraction is successful only when the channel capacity is higher than the number of bits that are embedded into the host image [ 28]. The watermarking capacity has been defined by the probability of detection, the probability of false alarm, and the mean square error. When more watermark data is inserted into the host image, more distortion is visible. However, distortion is not tolerable in military and medical applications. Watermarking techniques, therefore, must be implemented to minimize the distortion with less data embedding capacity. In this regard, the combination of IWT (Integer wavelets transform), the bit-plane method, and a QR (Quick Response) code has been proposed [ 29], where the watermark is converted into a QR code. Thus, the proposed method reduces the embedding capacity. 3.5. Computational Cost The computational cost for embedding a watermark into a host image and extracting the watermark from the watermarked image should be minimal. This cost includes two main issues—the total time required for embedding and extracting the watermark, and the total number of embedders and detectors involved in the watermarking technique. A good trade-o between robustness and computational complexity must be maintained. It has been implemented in reference [ 30] to ensure the security and robustness of microscopy images.Information 2020 ,11, 110 7 of 38 3.6. False Positive The false positive rate is the characteristic used to identify watermarks in an image where there is no watermark image. This problem occurs when the embedded watermark is di erent from the extracted watermark [ 31]. The test has been carried out by various schemes. This characteristic has mainly been used for copy control and ownership. If a watermark image Whas length land the extracted watermark is W’, then the false positive rate (FPR) is defined by the following equation [ 32]: FPR=l0 l(5) where l’is the Hamming distance of Wand W’. 3.7. Watermark Keys The watermark key is the secret key that determines certain parameters of the embedding function. This key includes the subset of image coe cients, the embedding direction, and /or the embedding domain. The estimation and mapping of the watermark key are important, as it determines the degree of security of the watermarking system and depends on certain parameters, such as the embedded message and watermarked image [ 33]. Therefore, for the embedding and extraction process, a secret key is needed to ensure security. The secret key includes a private key, a detection key, and a public key. The private key is available only to the user, the detection key is acknowledged in a court of law, and the public key is extracted by the public [ 34]. A study by Chopra et al. [ 35] used an Exclusive OR operation for the watermark key, such that that the system inserts the watermark into a defined location in the biometric signature template. The locations for di erent images are di erent from each other. This characteristic reduces the probability of the occurrence of various attacks. Therefore, the robustness of the system is increased. 3.8. Tamper Resistance Tamper-detection in the watermarking system can be used to check authenticity. Any change to the watermark data results in tampering of the image. Therefore, by testing integrity, the system determines whether the watermark data has been tampered or not [36]. 3.9. Reversibility The reversibility characteristic ensures the extraction of the watermark and exact reconstruction of the host image. However, for medical imaging, the modified image is used as a host image and the reconstructed image is used for diagnosis [ 37]. In the reversible digital watermarking method, the system takes the original image and obtains the watermarked image. Then, with the help of the extraction algorithm, the system recovers the original image and watermark image using the secret key. 3.10. Techniques that Meet Requirements Simultaneously From the above discussions, it can be summarized that it is impossible to satisfy imperceptibility, robustness, and capacity simultaneously due to their conflicting and limited characteristics [ 1]. For any watermarking system, imperceptibility may be decreased by increasing the properties of robustness and capacity, and vice versa [ 38]. On the other hand, robustness may be decreased by increasing the payload capacity. Therefore, a good trade-o among these types of requirements must be maintained. The trade-o among these three requirements is illustrated in Figure 3 [39]. However, there is no unique set of properties that are satisfied by all watermarking systems. Currently, some techniques exist that meet some of the above-discussed requirements simultaneously. These techniques are shown in Table 1. For example, in [ 40], a DCT, DWT, and SVD-based study, based on multiple watermarking techniques for securing online social network content, was proposed. A three-level DWT was applied to the host image for embedding the watermark, and a back-propagationInformation 2020 ,11, 110 8 of 38 neural network (BPNN) algorithm was applied to the extracted watermark image to minimize the distortion between the host image and the watermarked image. Thus, the robustness of the system was enhanced. For increasing security, multiple watermarks were embedded into the host image by using a selective encryption method. The experimental result showed superior performance over existing methods. In another study [ 41], a DCT, DWT, and SVD-based algorithm was proposed, which used multiple watermarking to ensure robustness, imperceptibility, capacity, and security simultaneously. To enhance security, the Arnold transform was applied to the host image before embedding. The quality of the watermarked image was satisfactory for diagnosis, in terms of human perception. Thus, the system ensures better imperceptibility at di erent gain factors. Phadikar et al. [ 42] proposed a reversible watermarking technique for Digital Imaging and Communications in Medicine (DICOM) images. The watermark was embedded into the host image in the lifting-based DWT domain. The experimental results showed that the technique ensures high embedding capacity along with better imperceptibility and robustness. Information 2020 , 11, x FOR PEER REVIEW 8 of 42 3.10. Techniques that Meet Requirements Simultaneously From the above discussions, it can be summarized th at it is impossible to satisfy imperceptibility, robustness, and capacity simultaneously due to their conflicting and limited characteristics [1]. For any watermarking system, imperceptibility may be decreased by increasing the properties of robustness and capacity, and vice versa [38]. On the other hand, robustness may be decreased by increasing the payload capacity. Therefore, a good trade-off among these types of requirements must be maintained. The trade-off among these three re quirements is illustrated in Figure 3 [39]. Figure 3. The trade-off among imperceptibility, robustness, and capacity. However, there is no unique set of properties that are satisfied by all watermarking systems. Currently, some techniques exist that meet some of the above-discussed requirements simultaneously. These techniques are shown in Tabl e 1. For example, in [40], a DCT, DWT, and SVD- based study, based on multiple watermarking techni ques for securing online social network content, was proposed. A three-level DWT was applied to the host image for embedding the watermark, and a back-propagation neural network (BPNN) algorithm was applied to the extracted watermark image to minimize the distortion between the host imag e and the watermarked image. Thus, the robustness of the system was enhanced. For increasing security, multiple watermarks were embedded into the host image by using a selective encryption method. The experimental result showed superior performance over existing methods. In another study [41], a DCT, DWT, and SVD-based algorithm was proposed, which used multiple watermarking to ensure robustness, imperceptibility, capacity, and security simultaneously. To enhance security , the Arnold transform was applied to the host image before embedding. The quality of the watermarked image was satisfactory for diagnosis, in terms of human perception. Thus, the system ensures better imperceptibility at different gain factors. Phadikar et al. [42] proposed a reversible watermarking technique for Digital Imaging and Communications in Medicine (DICOM) images. The watermark was embedded into the host image in the lifting-based DWT domain. The experimental results showed that the technique ensures high embedding capacity along with better imperceptibility and robustness. Table 1. Techniques that meet some requirements simultaneously. Used Techniques Factors Results Applications Hybrid transform domain and Particle Swarm Optimization (PSO) algorithm [15] Imperceptibility and robustness Performs better than existing methods Image Authentication APBT-based algorithm and SVD [22] Imperceptibility and robustness Better imperceptibility and good robustness Authenticity and integrity of copyright protection DWT, APDCBT, and SVD [23] Imperceptibility and robustness Performs better than existing methods Copyright protection Spatial domain technique [30] Robustness and computational complexity Guarantees of security and robustness Protection of Microscopy Images Figure 3. The trade-o among imperceptibility, robustness, and capacity. Table 1. Techniques that meet some requirements simultaneously. Used Techniques Factors Results Applications Hybrid transform domain and Particle Swarm Optimization (PSO) algorithm [15]Imperceptibility and robustnessPerforms better than existing methodsImage Authentication APBT-based algorithm and SVD [22]Imperceptibility and robustnessBetter imperceptibility and good robustnessAuthenticity and integrity of copyright protection DWT, APDCBT, and SVD [23]Imperceptibility and robustnessPerforms better than existing methodsCopyright protection Spatial domain technique [30]Robustness and computational complexityGuarantees of security and robustnessProtection of Microscopy Images BPNN [40]Robustness, security, and capacityBetter robustness and securityProtection of digital contents DWT, DCT, and SVD [41]Robustness, imperceptibility, capacity, and securityAcceptable visual quality for diagnosisHealthcare Lifting and companding [42]Imperceptibility, capacity, and securityPerforms better than existing reversible watermarking techniquesDICOM images 4. Digital Image Watermarking Applications Digital image watermarking is a highly focused research area, due to its potential use in media applications such as copyright protection, annotation, privacy control, data authentication, device control, media forensics, and medical reports (e.g., X-rays). Some associated applications of digital image watermarking are shown in Figure 4.Information 2020 ,11, 110 9 of 38 Information 2020 , 11, x FOR PEER REVIEW 9 of 42 BPNN [40] Robustness, security, and capacity Better robustness and security Protection of digital contents DWT, DCT, and SVD [41] Robustness, imperceptibility, capacity, and security Acceptable visual quality for diagnosis Healthcare Lifting and companding [42] Imperceptibility, capacity, and security Performs better than existing reversible watermarking techniques DICOM images 4. Digital Image Watermarking Applications Digital image watermarking is a highly focused re search area, due to its potential use in media applications such as copyright protection, annotation, privacy control, data authentication, device control, media forensics, and medical reports (e.g., X-rays). Some associated applications of digital image watermarking are shown in Figure 4. Figure 4. Related applications of digital image watermarking. The identified applications, according to design requirements, are shown in Table 2. Table 2. Design requirements and thei r corresponding applications. Requirements Applications Imperceptibility Copyright protection and fingerprinting. Robustness Copyright protection, content au thentication, and integrity verification. Security Copyright protection, data authentication, fingerprinting and tracking to digital contents, indexing, medical ap plications, and telemedicine data exchange. Capacity Tamper detection and integrity of medical images. Computational cost Protection of microscopy images. False positive Copy control and ownership. Watermark keys Copyright protection. Tamper resistance Authenticity and copyright integrity. Figure 4. Related applications of digital image watermarking. The identified applications, according to design requirements, are shown in Table 2. Table 2. Design requirements and their corresponding applications. Requirements Applications Imperceptibility Copyright protection and fingerprinting. Robustness Copyright protection, content authentication, and integrity verification. SecurityCopyright protection, data authentication, fingerprinting and tracking to digital contents, indexing, medical applications, and telemedicine data exchange. Capacity Tamper detection and integrity of medical images. Computational cost Protection of microscopy images. False positive Copy control and ownership. Watermark keys Copyright protection. Tamper resistance Authenticity and copyright integrity. Reversibility Medical applications. Some important and recent potential applications are described in the following subsections. 4.1. Broadcast Monitoring This application allows a content owner to verify when and where the content was broadcast. It also checks for the exact airtime of broadcasting content through satellite television and transmission media. Before the broadcast, a unique watermark can be inserted into each sound or video clip [ 8]. It is useful for several organizations and individuals when advertisers want to ensure that the content is broadcasted at exact airtime agreed by the customer and the advertisement company [ 43]. This application can be used to ensure the legal transmission of TV products such as news items [ 44]. In broadcast monitoring, the broadcast monitoring service provides the watermark data to the studio. The watermark data is embedded into the host media using the watermark embedding algorithm and the secret key. Then, the watermarked data is used by the studio. Finally, the TV station transmits this watermarked data as a TV program [ 45]. The basic framework of broadcast monitoring is shown in Figure 5.Information 2020 ,11, 110 10 of 38 Information 2020 , 11, x FOR PEER REVIEW 10 of 42 Reversibility Medical applications. Some important and recent potential applications are described in the following subsections. 4.1. Broadcast Monitoring This application allows a content owner to verify when and where the content was broadcast. It also checks for the exact airtime of broadcasting content through satellite television and transmission media. Before the broadcast, a unique watermark can be inserted into each sound or video clip [8]. It is useful for several organizations and individuals when advertisers want to ensure that the content is broadcasted at exact airtime agreed by the customer and the advertisement company [43]. This application can be used to ensure the legal transmission of TV products such as news items [44]. In broadcast monitoring, the broadcas t monitoring service provides the watermark data to the studio. The watermark data is embedded into the host media using the watermark embedding algorithm and the secret key. Then, the watermarked data is used by the studio. Finally, the TV station transmits this watermarked data as a TV program [45]. The basic framework of broadcast monitoring is shown in Figure 5. Figure 5. Broadcast monitoring using digital watermarking. 4.2. Copyright Protection, Ownership Assertion, or Owner Identification In copyright protection (or copyright holder identity) applications, a visible watermark identifies the copyright owner and ensures the proper payment of royalties. In this case, the owner can protect multimedia data by making the owners hip mark visible when be ing used commercially, if it is available on the internet [46]. An invisibl e and inseparable watermark is the best solution for identifying copyright ownership. This application proves ownership by extracting the embedded information from the watermarked document, compared to easily removable text marks. It requires strong robustness, such that the watermarked image cannot be removed without data distortion [44]. In reference [47], a fragile and robust watermark image was embedded into the host image for copyright protection and tampering detection. For copyright protection, the author’s logo was inserted into the host image as a robust waterm ark. However, an attacker may try to remove the tough watermark. The experimental results showed that the proposed system could extract the tough watermark even when half of the image was cropped. Thus, the author’s logo can be seen. A three- dimensional (3D) mesh watermarking scheme has been proposed by Hamidi et al. [48], which protects copyright information based on mesh sa liency and a wavelet transform. Wavelet analysis was performed on the original 3D mesh and the wavelet coefficients were obtained using mesh saliency. The watermark data was inserted into th e original 3D mesh using a quantization index modulation (QIM) technique and secret keys, and watermark extraction was done in a reverse manner. The method demonstrated better imperceptibility and good robustness. To ensure the secure storage and transmission of satellite imagery, digital image watermarking techniques play an important role. To ensure copyright protection, one study [49] proposed an SHA-3-based novel reversible invisible watermarking scheme, which uses the hash function and an adaptive prediction algorithm. Figure 5. Broadcast monitoring using digital watermarking. 4.2. Copyright Protection, Ownership Assertion, or Owner Identification In copyright protection (or copyright holder identity) applications, a visible watermark identifies the copyright owner and ensures the proper payment of royalties. In this case, the owner can protect multimedia data by making the ownership mark visible when being used commercially, if it is available on the internet [ 46]. An invisible and inseparable watermark is the best solution for identifying copyright ownership. This application proves ownership by extracting the embedded information from the watermarked document, compared to easily removable text marks. It requires strong robustness, such that the watermarked image cannot be removed without data distortion [ 44]. In reference [ 47], a fragile and robust watermark image was embedded into the host image for copyright protection and tampering detection. For copyright protection, the author’s logo was inserted into the host image as a robust watermark. However, an attacker may try to remove the tough watermark. The experimental results showed that the proposed system could extract the tough watermark even when half of the image was cropped. Thus, the author’s logo can be seen. A three-dimensional (3D) mesh watermarking scheme has been proposed by Hamidi et al. [ 48], which protects copyright information based on mesh saliency and a wavelet transform. Wavelet analysis was performed on the original 3D mesh and the wavelet coe cients were obtained using mesh saliency. The watermark data was inserted into the original 3D mesh using a quantization index modulation (QIM) technique and secret keys, and watermark extraction was done in a reverse manner. The method demonstrated better imperceptibility and good robustness. To ensure the secure storage and transmission of satellite imagery, digital image watermarking techniques play an important role. To ensure copyright protection, one study [ 49] proposed an SHA-3-based novel reversible invisible watermarking scheme, which uses the hash function and an adaptive prediction algorithm. 4.3. Copy Control and Finger Printing Copy control prevents people from making illicit copies of content. In this regard, a watermark can be used to restrict copying by informing hardware devices or software. In copy protection, a pirate knows the status of hidden messages, which is the real threat. The message contains “Copy No More,” “Copy Once,” or “Copy Never.” On the other hand, in fingerprinting or transaction tracking schemes, an innocent user cannot be framed by the collusion of pirates, and at least one pirate can be traced by the detector [ 50]. Similar to fingerprinting, which identifies an individual, transaction tracking uniquely identifies each copy of the work. The watermark accounts for the recipient of each legal dissemination of the work and it has been verified that invisible watermarking performs better, as compared to visible watermarking [51]. 4.4. Content Authentication and Integrity Verification Digital images can be modified with the help of widely available sophisticated image processing tools. For secure communication, information must be protected from unauthorized access—this property is known as integrity. A watermark verifies the authenticity of an image. Any significant modification of the image can also change the watermark. These changes can be detected [ 52], whichInformation 2020 ,11, 110 11 of 38 indicates that the data has been tampered with. The basic framework of content authentication and integrity verification is shown in Figure 6. Information 2020 , 11, x FOR PEER REVIEW 11 of 42 4.3. Copy Control and Finger Printing Copy control prevents people from making illicit copies of content. In this regard, a watermark can be used to restrict copying by informing hardware devices or software. In copy protection, a pirate knows the status of hidden messages, which is the real threat. The message contains “Copy No More,” “Copy Once,” or “Copy Never.” On the othe r hand, in fingerprinting or transaction tracking schemes, an innocent user cannot be framed by the co llusion of pirates, and at least one pirate can be traced by the detector [50]. Similar to fingerprinting, which identifies an individual, transaction tracking uniquely identifies each copy of the work. The watermark accounts for the recipient of each legal dissemination of the work and it has been veri fied that invisible watermarking performs better, as compared to visible watermarking [51]. 4.4. Content Authentication and Integrity Verification Digital images can be modified with the help of widely available sophisticated image processing tools. For secure communication, information must be protected from unauthorized access—this property is known as integrity. A watermark verifies the authenticity of an image. Any significant modification of the image can also change the watermark. These changes can be detected [52], which indicates that the data has been tampered with. The basic framework of content authentication and integrity verification is shown in Figure 6. Figure 6. Content authentication and integrity verification. In reference [53], a three-level image watermarking embedding technique has been proposed for integrity verification. The visible and invisible watermarks are embedded in the first two levels, respectively, and alpha channel watermarking in the third level is used for integrity verification. This scheme has been shown to detect the tampered regi ons successfully. Another study [54] has proposed a method which detects and localizes the tampered region. The experimental results demonstrated that the quality of the recovered image was high. 4.5. Indexing This technique uses comments and markers or ke y information, which is embedded as a watermark into videos, movies, and news items. Then , the technique retrieves the required data used by the search engine [36]. 4.6. Medical Applications Image watermarking can be applied to protect the copyright of medical images. Patient’s information can be protected from illegal access by watermarking techniques. These applications include medical imaging, telehealth, and telemedi cine, among others. Medical imaging visualizes tissues, organs, or some other parts of the body, by using information and communications technologies. Telehealth involves telesurgery, te lediagnosis, teleconferences, and other medical applications. Telemedicine connects specialists and patients separated by a physical distance [55]. Therefore, to ensure the confidentiality, authenti city, integrity, and availability associated with Figure 6. Content authentication and integrity verification. In reference [ 53], a three-level image watermarking embedding technique has been proposed for integrity verification. The visible and invisible watermarks are embedded in the first two levels, respectively, and alpha channel watermarking in the third level is used for integrity verification. This scheme has been shown to detect the tampered regions successfully. Another study [ 54] has proposed a method which detects and localizes the tampered region. The experimental results demonstrated that the quality of the recovered image was high. 4.5. Indexing This technique uses comments and markers or key information, which is embedded as a watermark into videos, movies, and news items. Then, the technique retrieves the required data used by the search engine [36]. 4.6. Medical Applications Image watermarking can be applied to protect the copyright of medical images. Patient’s information can be protected from illegal access by watermarking techniques. These applications include medical imaging, telehealth, and telemedicine, among others. Medical imaging visualizes tissues, organs, or some other parts of the body, by using information and communications technologies. Telehealth involves telesurgery, telediagnosis, teleconferences, and other medical applications. Telemedicine connects specialists and patients separated by a physical distance [ 55]. Therefore, to ensure the confidentiality, authenticity, integrity, and availability associated with Electronic Patient Record (EPR) data exchange, suitable watermarking techniques can be used. In these applications, the image quality must not be a ected by the watermark data [36]. 4.7. Other Applications Digital image watermarking can be used for proof of authenticity of the object’s originator. Additionally, digital counterfeiting, fraud, identify theft, secured electronic voting, and deployable remote education, among many others, are all possible applications of digital image watermarking. 5. Survey on Digital Image Watermarking Techniques Digital image watermarking has gained attention from researchers due to its availability and the delivery of redundant information. These techniques protect digital content from unauthorized access and manipulation. These techniques are required for di erent applications, such as authentication, operator acknowledgment, material security, and trademark protection. Digital image watermarking techniques can be classified based on the working domain, kinds of documents, nature of the algorithm, human perception, and type of application, as illustrated in Figure 7.Information 2020 ,11, 110 12 of 38 Information 2020 , 11, x FOR PEER REVIEW 12 of 42 Electronic Patient Record (EPR) data exchange, suitable watermarking techniques can be used. In these applications, the image quality must not be affected by the watermark data [36]. 4.7. Other Applications Digital image watermarking can be used for proof of authenticity of the object’s originator. Additionally, digital counterfeiting, fraud, identify theft, secured electronic voting, and deployable remote education, among many others, are all possible applications of digital image watermarking. 5. Survey on Digital Image Watermarking Techniques Digital image watermarking has gained attention from researchers due to its availability and the delivery of redundant information. These techni ques protect digital content from unauthorized access and manipulation. These techniques are required for different applications, such as authentication, operator acknowledgment, material security, and trademark protection. Digital image watermarking techniques can be classified based on the working domain, kinds of documents, nature of the algorithm, human perception, and type of application, as illustrated in Figure 7. Figure 7. Classification of image watermarking techniques. All digital image watermarking techniques depend on the type of working domain (i.e., spatial, frequency, or hybrid domain), type of documents (i .e., text, image, audio, or video), nature of the algorithm used (i.e., sequential or parallel), human perceptibility (i.e., visible or invisible), and type of application (i.e., source or destination-based). This section analyzes various digital image watermarking methods, based on the working domain, by summarizing some recent research results in this field. This section will be helpful for futu re studies on state-of-the-art watermarking methods. 5.1. Spatial Domain Watermarking Techniques This technique inserts watermark information into the host image, as defined by the owner in the spatial or time domain, using different methods including least significant bit (LSB) modification algorithms, intermediate significant bits (ISB) or patchwork algorithms, and spread spectrum and correlation-based algorithms. These techniques work directly on the original image pixels. The watermark can be inserted by manipulating the pixel values, based on a logo or signature information provided by the author [17]. In the most commonly used designs, pixel intensities at known points in space represent the image, where the lowest-order bi t of certain pixels in a color or grayscale image is flipped. Depending on the pixel intensity, the re sulting watermark may be visible or invisible. We review various approaches regarding spatial domain techniques that have attracted the attention of researchers due to their optimal balance among impe rceptibility, robustness, and capacity, which are Figure 7. Classification of image watermarking techniques. All digital image watermarking techniques depend on the type of working domain (i.e., spatial, frequency, or hybrid domain), type of documents (i.e., text, image, audio, or video), nature of the algorithm used (i.e., sequential or parallel), human perceptibility (i.e., visible or invisible), and type of application (i.e., source or destination-based). This section analyzes various digital image watermarking methods, based on the working domain, by summarizing some recent research results in this field. This section will be helpful for future studies on state-of-the-art watermarking methods. 5.1. Spatial Domain Watermarking Techniques This technique inserts watermark information into the host image, as defined by the owner in the spatial or time domain, using di erent methods including least significant bit (LSB) modification algorithms, intermediate significant bits (ISB) or patchwork algorithms, and spread spectrum and correlation-based algorithms. These techniques work directly on the original image pixels. The watermark can be inserted by manipulating the pixel values, based on a logo or signature information provided by the author [ 17]. In the most commonly used designs, pixel intensities at known points in space represent the image, where the lowest-order bit of certain pixels in a color or grayscale image is flipped. Depending on the pixel intensity, the resulting watermark may be visible or invisible. We review various approaches regarding spatial domain techniques that have attracted the attention of researchers due to their optimal balance among imperceptibility, robustness, and capacity, which are the most important requirements of any watermarking technique. These techniques have low complexity, improved e ciency, and faster execution. Furthermore, the watermarked image quality may be controlled [ 56]. However, these techniques perform well only if the image is not exposed to any noise or human modification. Picture cropping can be used to exclude the watermark, which is a major weakness in spatial domain watermarking. These techniques embed a large volume of data, in terms of capacity, but the inserted data may be easily detected by various attacks [ 57–59]. Additionally, a small object can be inserted several times. Hence, a single surviving watermark will be considered an achievement, despite losing most of the image due to several attacks. 5.1.1. Least Significant Bit (LSB) Least significant bit modification is the most commonly used algorithm for spatial domain watermarking. Here, the least significant bit (LSB) of randomly chosen pixels can be altered to hide the most significant bit (MSB) of another. It generates a random signal by using a specific key. The watermark is inserted into the least significant bits of the host image and can be extracted in the same way. Several techniques may process the host image. This type of algorithm is easy to implement and isInformation 2020 ,11, 110 13 of 38 simple. The least significant bits carry less relevant information and, thus, the quality of the host image is not a ected. It provides high perceptual transparency with a negligible impact on the host image. However, this algorithm can be a ected by undesirable noise, cropping, lossy compression, and so on, and may be attacked by a hacker by setting all the LSB bits to “1,” modifying the embedded watermark easily without any di culty. The LSB technique can easily be understood by the example depicted in Figure 8. Suppose two pixel values in the host image are 130 (10,000,010) and 150 (10,010,110). Then, using the LSB technique, if the embedded watermark is 10, then the watermarked pixel values will be 131 (10,000,011) and 150 (10,010,110), respectively. Information 2020 , 11, x FOR PEER REVIEW 13 of 42 the most important requirements of any watermarking technique. These techniques have low complexity, improved efficiency, and faster execution. Furthermore, the watermarked image quality may be controlled [56]. However, these techniques pe rform well only if the image is not exposed to any noise or human modification. Picture cropping can be used to exclude the watermark, which is a major weakness in spatial domain watermarking. These techniques embed a large volume of data, in terms of capacity, but the inserted data may be easily detected by various attacks [57–59]. Additionally, a small object can be inserted severa l times. Hence, a single surviving watermark will be considered an achievement, despite losing most of the image due to several attacks. 5.1.1. Least Significant Bit (LSB) Least significant bit modification is the most commonly used algorithm for spatial domain watermarking. Here, the least significant bit (LSB) of randomly chosen pixels can be altered to hide the most significant bit (MSB) of an other. It generates a random signal by using a specific key. The watermark is inserted into the least significant bits of the host image and can be extracted in the same way. Several techniques may process the host image. This type of algorithm is easy to implement and is simple. The least significant bits carry less rele vant information and, thus, the quality of the host image is not affected. It provides high perceptual transparency with a negligible impact on the host image. However, this algorithm can be affected by undesirable noise, cropping, lossy compression, and so on, and may be attacked by a hacker by setting all the LSB bits to “1,” modifying the embedded watermark easily without any difficulty. The LSB technique can easily be understood by the example depicted in Figure 8. Suppose two pixel values in the host image are 130 (10,000,010) and 150 (10,010,110). Then, using the LSB technique, if the embedded watermark is 10, then the watermarked pixel values will be 131 (10,000,011) and 150 (10,010,110), respectively. Figure 8. Basic least significant bit (LSB) technique example. Several researchers have studied modification s of the LSB technique, which are commonly related to the spatial domain. LSB techniques have been developed based on a bit-plane of digital discrete signals (e.g., audio or images). A bit-plane that represents the signal is a set of bits having the same bit position in each of the binary numb ers. Most techniques use only one bit-plane for embedding. This technique works on the least signifi cant bit (i.e., the eighth bit-planes), but others have used three bit-planes (i.e., the sixth–eighth bit-planes) or even four bit-planes (i.e., the fifth– eighth bit-planes) for embedding with acceptable imag e quality. The four least significant bits (i.e., the fifth–eighth bits of the cover image) can be replaced with the chosen bit of the secret image by simply using an OR operation in a specific manner [60]. This method first converts the host image into a stream of binary bits, outputs zero in the embedded bit, and then shifts the secret image to the right by 4 bits. Then, an OR operation is performed on these two (i.e., the host and secret images) to obtain the combined image. This oper ation is illustrated in Figure 9. Figure 8. Basic least significant bit (LSB) technique example. Several researchers have studied modifications of the LSB technique, which are commonly related to the spatial domain. LSB techniques have been developed based on a bit-plane of digital discrete signals (e.g., audio or images). A bit-plane that represents the signal is a set of bits having the same bit position in each of the binary numbers. Most techniques use only one bit-plane for embedding. This technique works on the least significant bit (i.e., the eighth bit-planes), but others have used three bit-planes (i.e., the sixth–eighth bit-planes) or even four bit-planes (i.e., the fifth–eighth bit-planes) for embedding with acceptable image quality. The four least significant bits (i.e., the fifth–eighth bits of the cover image) can be replaced with the chosen bit of the secret image by simply using an OR operation in a specific manner [60]. This method first converts the host image into a stream of binary bits, outputs zero in the embedded bit, and then shifts the secret image to the right by 4 bits. Then, an OR operation is performed on these two (i.e., the host and secret images) to obtain the combined image. This operation is illustrated in Figure 9. Information 2020 , 11, x FOR PEER REVIEW 14 of 42 Figure 9. Block diagram of the LSB method (four bit-planes). An example can best describe the above figure. Let the pixel value of the cover image (or host image) be 130 (10,000,010) and the binary representation of the secret image be 11,110,000. After embedding out zero, the cover image value is 128 (10,000,000). After shifting right by four bits, the secret image value is 15 (00,001,111). Then, an exclus ive-OR operation is done to obtain the combined image pixel, which has a decimal value of 143 (10,001,111). Therefore, this method shows the worst scenario when the host image and the secret image seem to be the same [61]. In other words, the difference between the host image and the secret image is (2 k-1), where k is the number of different bit-planes. To obtain random signals, a specific key (along with an m-sequence generator) can be used by the LSB algorithm. Thus, using the Huffman method, a two-dimensional watermark signal can be inserted into the host image with the correspondin g pixel value [2]. The method of Fung and Godoy [62] changes the host image pixels—only half of the bits (the least 1–4 bits), on average—with the number of bits of the embedded secret message. The human visual system cannot recognize this, du e of negligible changes in the intensity of colors. Yet, a submissive attacker can easily detect the changed bits, due to the simple operation involved. Manjula and Danti proposed a 2-3-3 LSB inse rtion method [63], which uses secret data that contains eight bits. These data are inserted into the LSB in a 2-3-3 order, such that two (02) bits are inserted into the R channel, three (03) bits are in serted into the G channel, and the remaining three (03) bits are inserted into the B channel. This method improves the MSE and PSNR values over the hash-based 3-3-2 technique. In a block-based meth od, the cover image may be processed by splitting it into blocks using certain techniques, such that the secret image can never be extracted. Then, the embedded watermark is encoded by modifying th e relationships between neighboring blocks. The conventional spatial domain watermarking technique has the highest probability of creating a salt-and-pepper noise effect. Hence, a method has been proposed by Abraham and Paul [64] for color image watermarking in the spatial domain without degrading image quality meaningfully and changing the perceptual color, as compared to conventional spatial domain watermarking. To make authentication and/or recovery po ssible, the watermark is embedded into all image blocks to ensure the higher quality of the image and high robustness against attacks. M1 and M2 ensure that the embedded bits are less disrupting to the human vi sual system, where M1 is the embedding mask and M2 is the compensation mask. The modified pixels are not noticeable, compared with neighboring pixels. Experimental results showed that their proposed algorithm recovered the watermark data even after the least significant bits were distorted and that the algorithm assured a good PSNR value. Although the LSB technique can easily be modifie d, understanding how the digital image will be modified, concerning integrity and safety, is a challenging task. The LSB hash algorithm authenticates the digital image using a hashing scheme which hides the hash function. One study embedded LSB hash code to protect the original file and extract the embedded hash code in order to produce an output file that appeared to be the same as the original file. In this case, the embedded watermark, Figure 9. Block diagram of the LSB method (four bit-planes). An example can best describe the above figure. Let the pixel value of the cover image (or host image) be 130 (10,000,010) and the binary representation of the secret image be 11,110,000. After embedding out zero, the cover image value is 128 (10,000,000). After shifting right by four bits, the secret image value is 15 (00,001,111). Then, an exclusive-OR operation is done to obtain the combined image pixel, which has a decimal value of 143 (10,001,111). Therefore, this method shows the worst scenario when the host image and the secret image seem to be the same [ 61]. In other words, theInformation 2020 ,11, 110 14 of 38 dierence between the host image and the secret image is (2 k-1), where kis the number of di erent bit-planes. To obtain random signals, a specific key (along with an m-sequence generator) can be used by the LSB algorithm. Thus, using the Hu man method, a two-dimensional watermark signal can be inserted into the host image with the corresponding pixel value [ 2]. The method of Fung and Godoy [ 62] changes the host image pixels—only half of the bits (the least 1–4 bits), on average—with the number of bits of the embedded secret message. The human visual system cannot recognize this, due of negligible changes in the intensity of colors. Yet, a submissive attacker can easily detect the changed bits, due to the simple operation involved. Manjula and Danti proposed a 2-3-3 LSB insertion method [ 63], which uses secret data that contains eight bits. These data are inserted into the LSB in a 2-3-3 order, such that two (02) bits are inserted into the R channel, three (03) bits are inserted into the G channel, and the remaining three (03) bits are inserted into the B channel. This method improves the MSE and PSNR values over the hash-based 3-3-2 technique. In a block-based method, the cover image may be processed by splitting it into blocks using certain techniques, such that the secret image can never be extracted. Then, the embedded watermark is encoded by modifying the relationships between neighboring blocks. The conventional spatial domain watermarking technique has the highest probability of creating a salt-and-pepper noise e ect. Hence, a method has been proposed by Abraham and Paul [ 64] for color image watermarking in the spatial domain without degrading image quality meaningfully and changing the perceptual color, as compared to conventional spatial domain watermarking. To make authentication and /or recovery possible, the watermark is embedded into all image blocks to ensure the higher quality of the image and high robustness against attacks. M1 and M2 ensure that the embedded bits are less disrupting to the human visual system, where M1 is the embedding mask and M2 is the compensation mask. The modified pixels are not noticeable, compared with neighboring pixels. Experimental results showed that their proposed algorithm recovered the watermark data even after the least significant bits were distorted and that the algorithm assured a good PSNR value. Although the LSB technique can easily be modified, understanding how the digital image will be modified, concerning integrity and safety, is a challenging task. The LSB hash algorithm authenticates the digital image using a hashing scheme which hides the hash function. One study embedded LSB hash code to protect the original file and extract the embedded hash code in order to produce an output file that appeared to be the same as the original file. In this case, the embedded watermark, which is used for extracting the data, is invisible [ 65]. However, LSB techniques can easily be implemented, and, thereby, the associated computational complexity may be reduced [8]. 5.1.2. Intermediate Significant Bit (ISB) LSB techniques are the most common and simple advanced watermarking techniques in the spatial domain, but they do not ensure robustness against attacks. For this reason, alternative methods, such as intermediate significant bit (ISB) methods, have been developed to improve the robustness and preserve the quality of the watermarking system. Several studies have developed ISB methods using dierent algorithms. One of these methods replaces the classic LSB technique with ISB by finding the best pixel value in between the middle and the edge of the range. In this method, the watermark image is protected from various attacks and alteration of watermarked image is minimized [ 66]. Another study [ 67] concentrated on the dual intermediate significant bit (DISB) model, in which two bits are embedded into each pixel of the host image and the remaining six (06) bits are changed to adjust the original pixel. The watermark image can be chosen by selecting the nearest pixel value to the original, if there exists a di erence between the original and the embedded one. The proposed model produces a higher quality watermarked image, as compared to LSB methods. Therefore, the DISB method ensures high robustness against attacks and improves the quality of the watermarked images. Robustness and quality are the two most essential requirements for any watermarking system, which can be analyzed by fair normalized cross-correlation (NCC) values.Information 2020 ,11, 110 15 of 38 ISB techniques have been used for image watermarking in the spatial domain. This technique substitutes original image pixels with watermark pixels by keeping the watermark pixels close to a filled or empty region in the original image pixels. The watermark pixel value is tested, according to the range of each bit-plane, and then the original image file pixel is placed outside any of the edges of the range [ 68]. There are eight (08) bit planes in grayscale images, where the first bit-plane holds the MSB while the eighth contains the LSB, and the remaining (second–seventh) bit-planes are used as ISB [ 69]. If the pixel value of a grayscale image is 133 (10,000,101), then intermediate significant bits are represented by Figure 10. Information 2020 , 11, x FOR PEER REVIEW 15 of 42 which is used for extracting the data, is invisible [65]. However, LSB techniques can easily be implemented, and, thereby, the associated computational complexity may be reduced [8]. 5.1.2. Intermediate Significant Bit (ISB) LSB techniques are the most common and simple advanced watermarking techniques in the spatial domain, but they do not ensure robustness against attacks. For this reason, alternative methods, such as intermediate significant bit (ISB) methods, have been developed to improve the robustness and preserve the quality of the watermarking system. Several studies have developed ISB methods using different algorithms. One of these methods replaces the classic LSB technique with ISB by finding the best pixel value in between the middle and the edge of the range. In this method, the watermark image is protected from various attacks and alteration of watermarked image is minimized [66]. Another study [67] concentrated on the dual intermediate significant bit (DISB) model, in which two bits are embedded into each pi xel of the host image and the remaining six (06) bits are changed to adjust the original pixel. The watermark image can be chosen by selecting the nearest pixel value to the original, if there exists a difference between the original and the embedded one. The proposed model produces a higher quality watermarked image, as compared to LSB methods. Therefore, the DISB method ensures hi gh robustness against attacks and improves the quality of the watermarked images. Robustness and quality are the two most essential requirements for any watermarking system, which can be analyzed by fair normalized cross-correlation (NCC) values. ISB techniques have been used for image watermarking in the spatial domain. This technique substitutes original image pixels with watermark pixels by keeping the watermark pixels close to a filled or empty region in the original image pixels . The watermark pixel value is tested, according to the range of each bit-plane, and then the original image file pixel is placed outside any of the edges of the range [68]. There are eight (08) bit planes in grayscale images, where the first bit-plane holds the MSB while the eighth contains the LSB, and the remaining (second–seventh) bit-planes are used as ISB [69]. If the pixel value of a grayscale image is 133 (10,000,101), then intermediate significant bits are represented by Figure 10. Figure 10. A bit-plane of a digital image. PSNR and NCC are the most frequently used quality metrics for watermarked images, where PSNR determines the intensities of strength and weakness of watermarking techniques, while the latter validates the strength of the used algorithm applied to a watermarked image after attacks. Considering this issue, the paper [69] revealed th e amount of strength and weakness of digital image w a t e r m a r k i n g t e c h n i q u e s b y d e f i n i n g t h e t h r e s h o l d v a l u e s o f P S N R a n d N C C . T h e d i s c u s s e d watermarking technique embedded four watermarks to the ISB of six grayscale image files one-by- one by substituting the original image pixels with new pixels and keeping them close to the original pixel values simultaneously. Their proposed algori thm demonstrated better robustness against some common image processing operations, such as filter ing, compression, noise, and blurring, based on the PSNR and NCC values. Robustness does not decr ease against geometric transformation attacks, such as scaling and rotation, in which pixel intensit ies are not be affected by their changed locations. Therefore, to increase the robustness of messages against various attacks an d to resist geometric Figure 10. A bit-plane of a digital image. PSNR and NCC are the most frequently used quality metrics for watermarked images, where PSNR determines the intensities of strength and weakness of watermarking techniques, while the latter validates the strength of the used algorithm applied to a watermarked image after attacks. Considering this issue, the paper [ 69] revealed the amount of strength and weakness of digital image watermarking techniques by defining the threshold values of PSNR and NCC. The discussed watermarking technique embedded four watermarks to the ISB of six grayscale image files one-by-one by substituting the original image pixels with new pixels and keeping them close to the original pixel values simultaneously. Their proposed algorithm demonstrated better robustness against some common image processing operations, such as filtering, compression, noise, and blurring, based on the PSNR and NCC values. Robustness does not decrease against geometric transformation attacks, such as scaling and rotation, in which pixel intensities are not be a ected by their changed locations. Therefore, to increase the robustness of messages against various attacks and to resist geometric transformations (e.g., scaling, cropping, and filtering), ISB techniques can be used, instead of LSB techniques, where the secret message can be embedded in a bit-plane (or bit-planes) [70]. 5.1.3. Patchwork Patchwork is a pseudo-random statistical process, which is embedded into an original image invisibly using redundant pattern encoding by a Gaussian distribution. Two patches A and B are chosen pseudo-randomly, and the image data of the first patch (A) are faded, while those in B are darkened. Patchwork methods show better robustness against maximum non-geometric image modifications and the process is independent of the content of the original image [ 71]. In this case, the robustness can be increased by either more a ne coding, feature recognition, or both, and the code can be lost by scaling, translation, or rotation before decoding. Although patchwork is impartially resistant to cropping, it does degrade its accuracy. The pseudo-random bitstream is generated by selecting pairs of pixels from the original image. A bit of information is encoded into the pair, where ddenotes the dierence between the two pixels; the encoding is 0 for d<0 and the pixels are swapped for d >0. The next pair can be progressed if dis equal to 0 or greater than a pre-defined threshold [ 72]. Therefore, the brightness can be increased by one unit at one point and decreased, respectively, at another point. This method is suitable for large areas of random texture, but not for text images. A region of random texture pattern in the image is copied to an area of the image with a similar texture. Each texture region is recovered through autocorrelation [ 73]. In a study by Yeo et al. [ 74], a generalized patchwork algorithm consisting of additive and multiplicative patchworks was proposed. This method uses statistical data to embed and detect the watermark. To detect the watermark data, this method usesInformation 2020 ,11, 110 16 of 38 the location-shift scheme and the scale-shift scheme. Their proposed method was shown to be robust against compression attacks. Yet, the robustness against various attacks is very high in the patchwork method; a small amount of data can be concealed [ 75]. The watermark can be embedded by using redundant pattern encoding into an image, and the watermark can be extracted using a secret key concerning the decoding algorithm. 5.2. Frequency (or Transform) Domain Watermarking Algorithms Spatial domain watermarking techniques are too fragile, as they can be easily manipulated. These techniques are much less robust against di erent types of attacks, compared to frequency-domain algorithms. These drawbacks have drawn focus to the research of transform-domain watermarking techniques which hide data in the transform space of a signal, rather than time, in a more e ective way. This technique converts an image using a pre-defined transform in order to represent the image in the frequency domain. Then, it embeds the watermark by changing the transform domain coe cients of the original image using di erent transforms, including the Discrete Cosine Transform (DCT), Discrete Fourier Transform (DFT), Discrete Wavelet Transform (DWT), Singular Value Decomposition (SVD), Hadamard, CAT, FFT, PHT, and Fresnel transform, among others. Finally, it extracts the watermark, with the help of a correct key, using an inverse transformation. Figure 11 describes the above procedure. Information 2020 , 11, x FOR PEER REVIEW 17 of 42 Figure 11. Watermark embedding and extracti on in the transform domain. To recover the original signal in the frequency domain, the frequency components must be recombined by applying phase shift information to each sinusoid of an image [1]. Many studies have been carried out on transform domain image watermarking, proving the better robustness, security, and imperceptibility against various attacks, such as compression, noise, filtering, cutting, and rotation. This section reviews some of those studies, which mostly used frequency-domain transforms, such as DCT, DFT, DWT, and SVD, and touches on hybrid domain methods. 5.2.1. Discrete Cosine Transform (DCT) The discrete cosine transform (DCT) separates an image into its equivalent frequency coefficients by modifying frequency components, which can be expressed as a sum of cosine functions. The DCT is a Fourier-related transform and contains a finite sequence of data points. Only real numbers can be used here. Its variance determ ines the usefulness of the DCT coefficients. The DCT is important for image compression; for instance, in the JPEG image format. The one-dimensional (1D) DCT is defined by the following equation [76]: 𝑦 (𝑘)= 𝛼 (𝑘)෍𝑥 (𝑛)cos(𝜋(2𝑛 + 1 )𝑘 2𝑁ேିଵ ௡ୀ଴), 𝑘 = 0, 1, … , 𝑁 − 1 (6) and the inverse transform is given by 𝑥(𝑛)= ෍𝛼 (𝑘)𝑦(𝑛)cos (𝜋(2𝑛 + 1 )𝑘 2𝑁ேିଵ ௞ୀ଴), 𝑛 = 0,1, … , 𝑁 − 1 (7) with 𝛼(0)= ටଵ ே,𝑘 =0 and 𝛼(𝑘)= ටଶ ே,1≤𝑘 ≤𝑁−1 (8) where N is the number of given data samples: x(0), …, x(N-1 ), x(n) is the input data sample, y(k) is the DCT coefficient, and α(k) is the scaling factor. Many studies have already been carried out on digital image watermarking methods in the DCT domain. Among these, block-based DCT image watermarking works by dividing the host image into different image blocks, following which the method applies the DCT transform to this image. Then, the method inserts the watermark into the block and DCT-based host image with the help of an algorithm. The inverse discrete cosine transform (IDCT) is then applied to obtain the watermarked Figure 11. Watermark embedding and extraction in the transform domain. To recover the original signal in the frequency domain, the frequency components must be recombined by applying phase shift information to each sinusoid of an image [ 1]. Many studies have been carried out on transform domain image watermarking, proving the better robustness, security, and imperceptibility against various attacks, such as compression, noise, filtering, cutting, and rotation. This section reviews some of those studies, which mostly used frequency-domain transforms, such as DCT, DFT, DWT, and SVD, and touches on hybrid domain methods. 5.2.1. Discrete Cosine Transform (DCT) The discrete cosine transform (DCT) separates an image into its equivalent frequency coe cients by modifying frequency components, which can be expressed as a sum of cosine functions. The DCT is a Fourier-related transform and contains a finite sequence of data points. Only real numbers can be used here. Its variance determines the usefulness of the DCT coe cients. The DCT is important forInformation 2020 ,11, 110 17 of 38 image compression; for instance, in the JPEG image format. The one-dimensional (1D) DCT is defined by the following equation [76]: y(k)=(k)N1X n=0x(n)cos((2n+1)k 2N),k=0, 1,:::,N1 (6) and the inverse transform is given by x(n)=N1X k=0(k)y(n)cos((2n+1)k 2N),n=0, 1,:::,N1 (7) with (0)=r 1 N,k=0 and(k)=r 2 N, 1kN1 (8) where Nis the number of given data samples: x(0),:::, x(N1),x(n) is the input data sample, y(k)is the DCT coe cient, and(k)is the scaling factor. Many studies have already been carried out on digital image watermarking methods in the DCT domain. Among these, block-based DCT image watermarking works by dividing the host image into dierent image blocks, following which the method applies the DCT transform to this image. Then, the method inserts the watermark into the block and DCT-based host image with the help of an algorithm. The inverse discrete cosine transform (IDCT) is then applied to obtain the watermarked image. The above-discussed DCT methods for watermark embedding can be best described by Figure 12. Note that watermark extraction can be done in a reverse way. Information 2020 , 11, x FOR PEER REVIEW 18 of 42 image. The above-discussed DCT methods for watermar k embedding can be best described by Figure 12. Note that watermark extraction can be done in a reverse way. Figure 12. Watermark embedding in a block-based DCT domain. The method proposed by Kitanovski [77] perfor ms a block-based DCT transform in each of the blocks after dividing an image into P × P blocks and embeds a watermark generated using an image hash as a key in the low-frequency components using quantization index modulation (QIM). In QIM, only one watermark bit is inserted into each DCT block. Their proposed meth od demonstrated better robustness in image authentication [78]. Another pa per [12] introduced a method which was robust against JPEG compression along with a Gaussian low pass filter. This method embeds 4096 bits of information of an image whose dimension is 512 × 512 pixels, where adaptive quantization can be used to select the twelve representative DCT coefficients which embed the watermark. A Chinese Remainder Theorem (CRT)-based watermarking sche me in the DCT domain has been proposed by Jagdish C. Patra, which performs better against brightening, sharpening effects, and JPEG compression, as compared to a CRT scheme based on spatial domain methods, in terms of robustness and security [79]. Although DCT techniques are robust and resistant against common image processing operations, they require huge amounts of calculation. This is difficult to implement and shows weak performance against geometric transformation attacks, such as scaling, rotation, and cropping. A DCT/IDCT method has been proposed for ensuring effectiveness [80], in which a digital watermarking encryption algorithm was introduced. For authentication, integrity verification, tamper detection, and protection of digital data , a semi-blind robust DCT watermarking approach has been proposed which uses DCT and linear interp olation techniques [81], which divides the host image into N × N (usual blocks of 8×8) pixel blocks, as well as obtaining the corresponding DCT block, and calculates the inverse transform. In this cas e, the medium-frequency (MF) components can be used, such that a compromise between robustness and watermark visibility can be achieved. The study demonstrated the high robustness of the system against rotational attacks, JPEG compression attacks, noising attacks, and median filtering attacks. At this point, the system can extract the watermark correctly, which was the main contribution of the paper. The studies of Roy et al. [82] presented a DCT-based color watermarking technique for embedding multiple watermarks, designed for copyright ownership and validation. The system demonstrated better robustness and imperceptibility and generated a higher PSNR value by eliminating the main drawback—namely, blocking artifacts (loss of some information)—o f block-based DCT methods. One watermark bit was preserved by using an error-correcting code (ECC). However, the system exhibited high computational complexity. A study of Liu et al. [83] presented an improved DCT encryption method for watermarking, where the first encryption of host image is done by fractal encoding, while the second encryption is performed using DCT. This dual encryption method made the proposed system more robust and effective. A differential evolutio n and kernel extreme lear ning machine (DE-KELM)- based grayscale image watermarking method in th e DCT domain has been presented, where the low- frequency coefficients are selected in a zig-zag manner, such that the watermarked image quality is Figure 12. Watermark embedding in a block-based DCT domain. The method proposed by Kitanovski [ 77] performs a block-based DCT transform in each of the blocks after dividing an image into P P blocks and embeds a watermark generated using an image hash as a key in the low-frequency components using quantization index modulation (QIM). In QIM, only one watermark bit is inserted into each DCT block. Their proposed method demonstrated better robustness in image authentication [ 78]. Another paper [ 12] introduced a method which was robust against JPEG compression along with a Gaussian low pass filter. This method embeds 4096 bits of information of an image whose dimension is 512 512 pixels, where adaptive quantization can be used to select the twelve representative DCT coe cients which embed the watermark. A Chinese Remainder Theorem (CRT)-based watermarking scheme in the DCT domain has been proposed by Jagdish C. Patra, which performs better against brightening, sharpening e ects, and JPEG compression, as compared to a CRT scheme based on spatial domain methods, in terms of robustness and security [ 79]. Although DCT techniques are robust and resistant against common image processing operations, they requireInformation 2020 ,11, 110 18 of 38 huge amounts of calculation. This is di cult to implement and shows weak performance against geometric transformation attacks, such as scaling, rotation, and cropping. A DCT /IDCT method has been proposed for ensuring e ectiveness [80], in which a digital watermarking encryption algorithm was introduced. For authentication, integrity verification, tamper detection, and protection of digital data, a semi-blind robust DCT watermarking approach has been proposed which uses DCT and linear interpolation techniques [ 81], which divides the host image into N N (usual blocks of 8 8) pixel blocks, as well as obtaining the corresponding DCT block, and calculates the inverse transform. In this case, the medium-frequency (MF) components can be used, such that a compromise between robustness and watermark visibility can be achieved. The study demonstrated the high robustness of the system against rotational attacks, JPEG compression attacks, noising attacks, and median filtering attacks. At this point, the system can extract the watermark correctly, which was the main contribution of the paper. The studies of Roy et al. [ 82] presented a DCT-based color watermarking technique for embedding multiple watermarks, designed for copyright ownership and validation. The system demonstrated better robustness and imperceptibility and generated a higher PSNR value by eliminating the main drawback—namely, blocking artifacts (loss of some information)—of block-based DCT methods. One watermark bit was preserved by using an error-correcting code (ECC). However, the system exhibited high computational complexity. A study of Liu et al. [ 83] presented an improved DCT encryption method for watermarking, where the first encryption of host image is done by fractal encoding, while the second encryption is performed using DCT. This dual encryption method made the proposed system more robust and e ective. A di erential evolution and kernel extreme learning machine (DE-KELM)-based grayscale image watermarking method in the DCT domain has been presented, where the low-frequency coe cients are selected in a zig-zag manner, such that the watermarked image quality is not compromised [ 84]. Singh [ 85] solved the false positive detection problem which arises in the spatial domain by transforming the host image in the DCT domain, where non-overlapping blocks are generated from the DCT coe cients. These blocks create the circulant matrix, which embeds the watermark. Their proposed method extracts the watermark by generating dynamic stochastic resonance (DSR) phenomena, ensuring imperceptibility and robustness against conventional attacks. A chaotic encryption-based blind digital image watermarking technique has been proposed, which works both for grayscale and color images [ 24]. The method divides the host image into 88 blocks after performing DCT operation and, then, embeds the watermark using the DCT coe cients of adjacent blocks. To add another layer of security, Arnold transforms along with a chaotic map are used at this time. The results demonstrated the robustness of the system against common image processing operations. From the above studies, we may conclude that image watermarking is resistant against most attacks when using embedding in the DCT domain. However, it is susceptible to cropping and scaling [ 65]. Additionally, the DCT-based transform shows better results in concentrating energy into lower-order coe cients than the discrete Fourier transform (DFT) for image data. 5.2.2. Discrete Fourier Transform (DFT) The discrete Fourier transform (DFT) uses samples that are uniformly spaced. In this case, a sequence of fixed length numbers of uniformly spaced samples of a function is converted into a sequence of the same length of uniformly spaced samples in the discrete-time Fourier transform (DTFT). The DTFT uses a set of harmonically related complex (magnitude and phase) exponential functions. The DFT represents the original input sequence in the frequency domain and produces a signal that is discrete and periodic. Many practical applications, including signal processing, image processing, filters, convolution operations, spectrum analysis of sinusoids, and Fourier analysis, are done by DFT [86]. The one-dimensional (1D) DFT can be defined by the following equation [76]: y(k)=1p NN1X n=0x(n)exp j2 Nkn ,k=0, 1,:::,N1 (9)Information 2020 ,11, 110 19 of 38 The inverse transform is given by x(n)=1p NN1X k=0y(k)exp j2 Nkn ,n=0, 1,:::,N1 (10) with j=p 1 (11) where Nis the number of given data samples: x(0),:::, x (N1), y(k) is the DFT coe cient, and x(n) is the input data sample. Many studies have been carried out on the DFT for image watermarking. Tsui et al. [ 87] proposed two algorithms for color image watermarking that use multidimensional Fourier transforms. The spatial-chromatic discrete Fourier transform was used for embedding a color watermark (yellow and blue). A color image contains chromatic content, which is converted to CIE chromaticity co-ordinates in the frequency domain in this method. Then, the color watermark is inserted into the host image. Another method uses the quaternion Fourier transform to insert a watermark in the frequency domain after encoding the components of a color image. The experimental results demonstrate that the imperceptibility is maximized, and better robustness is exhibited against external attacks and many digital signal processing operations, as compared to other existing algorithms. The method shows the strength of the watermark is best. The DFT approach has demonstrated a strong robustness to geometric attacks, as well, as it is translation invariant. Therefore, a robust, simple, and fast watermarking method based on DFT with an optimal implementation radius has been studied by Poljicak et al. [ 88], where the watermark was embedded in the magnitude spectrum of the Fourier transform and the quality degradation of the watermarked image was shown to be minimal, by evaluating the PSNR. Their results demonstrated significant robustness against amplitude modulation, the print-scan process (PS), half-toning, the print-cam process (PC), cropping, and attacks from the StirMark benchmark software. Cedillo-Hernandez et al. [ 89] studied a robust watermarking method in the DFT domain for managing medical images, which ensured high robustness and preserved the quality of the watermarked images. The proposed method inserts the watermark into the magnitude spectrum of the middle frequencies of the DFT of the original medical image. The corresponding electronic patient record (EPR) data cannot be corrupted or detached from the medical images. It was shown that the system is robust against signal processing operations and geometric distortions. The performance of a watermarking technique is estimated in terms of robustness, imperceptibility, capacity, and detachment detection. Another paper [ 90] proposed the combination of the QDFT and a log-polar transform, where the QDFT is the quaternion discrete Fourier transform. The QDFT deals with the three channels (red, green, and blue) of color images. At first, the secondary image is computed using a log-polar transform. Then, the image is extracted from the low-frequency components of QDFT. This algorithm uses a secret key to enhance security. The proposed system is sensitive to alterations of image content. This method preserves content and ensures robustness, especially against rotation operations. Studies of DFT-based methods have shown that there exists a conflict problem between the quality and robustness of the systems. For this, a solution to this problem, based on the Fourier transform and characteristics of the visual system, has been presented [ 91], in which the host image is split into the blocks that do not overlap, and the watermark bits are embedded (inserted) within the selected coe cients of each block by executing certain conditions. Di erent types of attacks, such as gamma noise, Gaussian noise, sharpness, blurring, and filtering, can be minimized by this method, which exhibits better robustness. A DFT-based semi-fragile watermarking method with a substitution box has been presented by Jamal et al. [ 92], which embeds watermark bits generated by a chaotic map into the host image. Although this method is complex to compute, it has demonstrated improved robustness and security against di erent kinds of attacks. Therefore, these methods provide better robustness against geometric attacks (e.g., translation, rotation, scaling, and cropping), which makes DFT domain-based techniques a popular area of research. In this context, two types of DFT-basedInformation 2020 ,11, 110 20 of 38 watermark embedding techniques have been proposed. The first type inserts the watermark directly by changing phase information within the DFT. The second type is based on a template to judge the transformation factor in the DFT domain. Finally, a detector can be used to detect the embedded spread spectrum watermark [93]. 5.2.3. Discrete Wavelet Transform (DWT) In mathematics, a discrete wavelet transform (DWT) is any wavelet transform that decomposes a signal into wavelets, rather than frequencies. In a DWT, the wavelets are discretely sampled. The temporal resolution is one of the advantages of DWT over Fourier transforms (i.e., DCT and DFT). This makes DWT a more attractive research area, by capturing multiple information aspects, such as location in time and frequency [ 94]. A set of wavelets, which are mathematical functions, is used to decompose the signal. The wavelet transform is useful in digital signal processing, image compression, and in removing noise from the signal. The key idea in a wavelet transform is the use of a set of basis functions (called wavelets) that o er localization in the frequency domain. High frequency resolution can be obtained at low frequencies, and high time resolution can be obtained at high frequencies when using a wavelet transform. The DWT of a signal x[n] is defined by the following equations [95]: W[j0,k]=1p MX nx[n]j0,k[n] (12) W [j,k]=1p MX nx[n] j,k[n],f or jj0 (13) where W[j0,k]are the approximation coe cients, W [j,k]are the detail coe cients, and the inverse DWT is given by x[n]=1p MX kW[j0,k]j0,k[n]+1p MJX j=j0X kW [j,k] j,k[n] (14) with n=0, 1, 2,:::,M1,j=0, 1, 2,:::,J1,k=0, 1, 2,:::, 2j1 (15) where M is the number of samples to be transferred =2J, J is the number of transfer levels,n j,k[n]o and j,k[n]are two basis functions, [n]denotes the scaling function, and [n]denotes the wavelet function. The basic DWT image watermarking technique decomposes the original image into three di erent levels. The sub-bands LH3, HH3, and HL3, at three di erent levels, are used to embed the watermark. Sub-bands consist of a wide range of the frequency spectrum of the image. Therefore, the robustness of the watermarking system is increased [96]. A three-level DWT is shown in Figure 13. After applying the DWT, the system embeds the watermark into the host image by using an algorithm and, then, applies the inverse DWT (IDWT) to obtain the watermarked image. The watermark extraction process takes the watermarked image as input and applies the DWT at the same level. Finally, the process applies the IDWT to get the watermark image. The whole process is depicted in the Figure 14.Information 2020 ,11, 110 21 of 38 Information 2020 , 11, x FOR PEER REVIEW 21 of 42 𝑥[𝑛]= 1 √𝑀෍𝑊 థ[𝑗଴,𝑘]𝜙௝బ,ೖ[ 𝑛]+1 √𝑀 ௞෍෍𝑊 ట[𝑗, 𝑘] ௞௃ ௝ୀ ௝ బ𝜓௝ ,ೖ[𝑛] (14) with 𝑛 = 0, 1, 2, … , 𝑀 − 1 , 𝑗 = 0, 1, 2, … , 𝐽−1, 𝑘 = 0, 1, 2, … , 2௝−1 (15) where M is the number of samples to be transferred = 2J, J is the number of transfer levels, {𝜙௝,ೖ[𝑛]} and 𝜓௝ ,ೖ[𝑛] are two basis functions, ϕ[n] denotes the scaling function, and ψ[n] denotes the wavelet function. The basic DWT image watermarking technique decomposes the original image into three different levels. The sub-bands LH3, HH3, and HL3, at three different levels, are used to embed the watermark. Sub-bands consist of a wide range of the frequency spectrum of the image. Therefore, the robustness of the watermarking system is increase d [97]. A three-level DWT is shown in Figure 13. Figure 13. Three-level discrete wavelet decomposition. After applying the DWT, the system embeds the watermark into the host image by using an algorithm and, then, applies the inverse DWT (IDWT) to obtain the watermarked image. The watermark extraction process takes the watermar ked image as input and applies the DWT at the same level. Finally, the process applies the IDWT to get the watermark image. The whole process is depicted in the Figure 14. Figure 14. Watermark embedding and extraction in a Discrete Wavelet Transform (DWT) domain. Figure 13. Three-level discrete wavelet decomposition. Information 2020 , 11, x FOR PEER REVIEW 21 of 42 𝑥[𝑛]= 1 √𝑀෍𝑊 థ[𝑗଴,𝑘]𝜙௝బ,ೖ[ 𝑛]+1 √𝑀 ௞෍෍𝑊 ట[𝑗, 𝑘] ௞௃ ௝ୀ ௝ బ𝜓௝ ,ೖ[𝑛] (14) with 𝑛 = 0, 1, 2, … , 𝑀 − 1 , 𝑗 = 0, 1, 2, … , 𝐽−1, 𝑘 = 0, 1, 2, … , 2௝−1 (15) where M is the number of samples to be transferred = 2J, J is the number of transfer levels, {𝜙௝,ೖ[𝑛]} and 𝜓௝ ,ೖ[𝑛] are two basis functions, ϕ[n] denotes the scaling function, and ψ[n] denotes the wavelet function. The basic DWT image watermarking technique decomposes the original image into three different levels. The sub-bands LH3, HH3, and HL3, at three different levels, are used to embed the watermark. Sub-bands consist of a wide range of the frequency spectrum of the image. Therefore, the robustness of the watermarking system is increase d [97]. A three-level DWT is shown in Figure 13. Figure 13. Three-level discrete wavelet decomposition. After applying the DWT, the system embeds the watermark into the host image by using an algorithm and, then, applies the inverse DWT (IDWT) to obtain the watermarked image. The watermark extraction process takes the watermar ked image as input and applies the DWT at the same level. Finally, the process applies the IDWT to get the watermark image. The whole process is depicted in the Figure 14. Figure 14. Watermark embedding and extraction in a Discrete Wavelet Transform (DWT) domain. Figure 14. Watermark embedding and extraction in a Discrete Wavelet Transform (DWT) domain. Many studies have been carried out on the authentication of images in the DWT domain. For example, Chen [ 97] proposed a digital image watermarking algorithm based on threshold classification in the wavelet domain. The algorithm analyzes the complexity of the images concerning robustness and imperceptibility. The method divides the host image into di erent blocks, which are selected for embedding the watermark. Then, with the help of the derived optimal sub-bands, the DWT coe cient is classified based on sub-bands that have lower frequencies. Extensive experiments were carried out, and the results demonstrated strong robustness and imperceptibility of the system against some common attacks. Another paper [ 98] proposed an e cient digital image watermarking technique based on the wavelet transform in HSI color space for protecting copyright holder information, in which a one-level wavelet coe cient (LL) is generated in the host image, and 8 8 blocking operations are used on the grayscale watermark image. For embedding, each block of both images is compared and scaled with a scaling factor . At the receiver site, the system extracts the watermark image in the opposite manner. The simulation results demonstrated that their proposed scheme is more robust against noise, as compared to existing methods, in terms of PSNR and MSE. A di erent paper by Jia et al. [ 99] extracted the watermark without the pre-requisite of the original host image or the original watermark image. A combination of DWT and QR decomposition-based color image watermarking scheme was suggested, where one-level DWT is applied to each component of the host image. The color host image is divided into 4 4 non-overlapping blocks and QR decomposition is used to decompose each selected pixel block. In this way, the watermark can be embedded by measuring the first-row elements in the matrix R. The proposed method has better robustness against the addition of noise, image compression, cropping, and filtering, as compared to related existing methods. ForInformation 2020 ,11, 110 22 of 38 copyright protection, a new algorithm was studied in the DWT domain [ 69]. This blind scheme embeds the logo watermark directly into the three-level DWT decomposed sub-bands. Two kinds of security attacks were considered to confirm the security of the system. The simulation results confirmed blind detection, invisibility, and robustness against various geometric and non-geometric attacks. Another study [ 100] presented a new watermarking method in the DWT domain, based on a discrete-time chaotic system. The feasibility and robustness of the proposed method were demonstrated. The host image is encrypted by a Henon map and inserted into a discrete-time chaotic system. The resulting ciphered image can be used as a watermark image, and the extracted watermark is passed through the previous chaotic channel for decryption. The proposed technique by Ambadekar et al. [ 101] was designed to protect copyright information by inserting the watermark in the DWT domain and extracting the watermark through watermark encryption. This method exhibited strong robustness against noise, geometric, and compression attacks. As spatial domain-based image watermarking cannot resist geometric attacks, the transform domain has been considered a more attractive research area. Hence, a new approach has been designed, which can e ectively embed a color image into a host image. The technique transforms the color host image and watermarked image from the RGB model to the YIQ model, separately, and the two-dimensional DWT and corresponding selection algorithm are applied. This algorithm ensures the system’s robustness against attacks such as lossy compression and Gaussian noise. The method has been applied to ensure image security [102]. 5.2.4. Singular Value Decomposition (SVD) In mathematics, a singular value decomposition (SVD) is the product of a real or complex matrix. This method is the generalization of the eigendecomposition of a symmetric matrix with non-negative eigenvalues to any mnmatrix through an extension of the polar decomposition. The SVD transformation has been widely used in statistics and digital signal processing. The SVD of a matrix Mcan be defined by the following equation [103]: M=USV(16) where Mis an mnmatrix that comes from the field K(either real numbers or complex numbers), U is an mmunitary matrix over K, or an orthogonal matrix (if K=R),Pis a diagonal mnmatrix with non-negative real numbers on the diagonal, Vis an nnunitary matrix over K, and V*is the conjugate transpose of V. In the SVD, the diagonal entries iofPare known as the singular values of M. The one-way and non-symmetric properties cannot be obtained with the above-discussed DCT, DFT, and DWT transformations. Therefore, another transformation, using SVD, has been proposed for watermarking [ 104]. This method embeds the watermark by discovering the features of D and the relationship of the U component coe cients. This technique can extract the watermark e ciently, even after tampering. Simulation results demonstrated the high quality of the watermarked image and stronger robustness against various attacks, as compared to existing methods. Vaishnavi [ 105] proposed a method designed for robust and invisible image watermarking in the RGB color space. The method embeds the watermark into the singular values. The watermark is retrieved by engaging SVD on the blue channel of the host image. The proposed method was shown to provide better robustness against Gaussian noise, motion blur, salt-and-pepper noise, median filtering attacks, and JPEG compression attacks, among others, as compared to existing methods, where performance was evaluated using the normalized correlation (NC) and PSNR measures. The existing block-based SVD is not robust against geometric distortions, rotation, and image flipping. Therefore, an alternative solution has been proposed [ 106] that combines the concept of redistributed image normalization and SVD, where the coe cient magnitude is inserted. The proposed scheme is e cient against various attacks, in terms of robustness and security. Another algorithm has been proposed [ 107], based on SVD and homomorphic transform. This method ensures the digital security of an eight-bit grayscale imageInformation 2020 ,11, 110 23 of 38 by embedding an invisible eight-bit grayscale watermark image. The SVD finds the singular values in which the watermark is inserted. The robustness and invisibility of this watermarking system have been guaranteed by evaluating the PSNR, normalized cross correlation (NCC), and mean-structural similarity-index-measure (MSSIM) performance. 5.3. Hybrid Domain Watermarking Algorithms and Current Trends in Watermarking Hybrid domain watermarking algorithms are usually considered as a combination of spatial- and transform-domain algorithms. These algorithms ensure both robustness and enhanced data embedding properties. Many studies have been carried out on hybrid domain methods. These studies reflect the current trends in watermarking. For example, the authors of reference [ 108] combined the spatial and frequency domains for image watermarking, such that more watermark data could be embedded into the host image. This method increases the capacity of the host image and splits the watermark into two parts, which doubles the protection. The spatial domain substitutes the LSB bits of the host image with the bits of the watermark image. On the other hand, the frequency domain inserts data into the low-frequency components of the host image. Furthermore, a random permutation of the watermark can be used to enhance robustness against various signal processing attacks, such as image cropping. In reference [ 109], DCT, DWT, and SVD were combined to achieve better robustness and imperceptibility. This hybrid scheme was shown to be robust against JPEG compression, filtering, salt-and-pepper noise, rotation, cropping, and scaling and translation operations. Another scheme [ 110] was introduced to protect digital rights in a hybrid domain. The scheme protects digital contents during broadcast over a non-secure channel by using the least significant bits and a wavelet transform (DWT and SVD), where the host image is divided into sub-bands (LL, HL, HH, and LH) by using the transformation method in the frequency domain. The previously defined embedding algorithms are used to extract the watermark. This hybrid scheme provides better quality and robustness against dierent attacks, such as Gaussian noise and JPEG compression. Another study combined DCT and DWT to improve the robustness of the watermarking system [ 111]. A hybrid scheme of DWT and SVD has been proposed, which ensures both the robustness and imperceptibility of the watermarking system [ 112]. Existing watermarking algorithms are less robust to geometric attacks. Hence, to resist geometric attacks, a multi-watermarking algorithm has been proposed for medical images, based on dual-tree complex wavelet transform (DTCWT), DCT and a Henon Map [ 113]. This algorithm can be used for medical security, security authentication, cloud storage, and cloud transmission. 5.4. Summary of Watermarking Techniques of Working Domain From the above-discussed research, we can conclude that some watermarking algorithms are simple to implement. Some algorithms minimally degrade the image quality, while others highly distort the image. Some are complex to achieve, and some are highly robust against common image processing operations but are not resistant to geometric attacks. Then again, others are highly robust to geometric attacks but very sensitive to various kinds of noise. Some of them ensure better PSNR and NCC values, while others result in low PSNR. Based on the desired applications, some are robust, while others are fragile. Hence, this section summarizes the findings of the above-discussed state-of-the-art watermarking techniques among current trends in watermarking techniques, with the help of the following Table 3. Table 3 includes techniques in use, image type, image size in pixels, factors, advantages, limitations, and associated applications. Based on Table 3, we conclude that DCT provides high robustness and imperceptibility, while LSB exhibits the least robustness and imperceptibility. The ranking, in terms of robustness and imperceptibility, can be written as DCT>SVD>DWT>DFT>LSB. (17)Information 2020 ,11, 110 24 of 38 Table 3. Summary of state-of-the-art watermarking techniques (continued). Used Techniques Image TypeImage Size (pixels) (Host Image and Watermark Image Respectively)Factors Advantages Limitations Applications LSB Modification [64]Color512512, 6464RobustnessHigh quality of the watermarked image -High robustness to attacks -Good PSNR (47.6dB) -Fast speedThe worst scenario for having no di erence between the host image and the watermark image -Only B component is used for embedding colorCopyright protection LSB hash algorithm [65]- - CapacityExtract watermark data eectivelyLess robust to various attacksHistogram analysis, Hamming distance ISB [66] Grayscale256256, 9090RobustnessImprove robustness -Minimum distortion of the watermarked imageLess Robust against geometric attacks, like scaling, rotation, filtering, and cropping.Image authentication DISB [67] Grayscale256256, -Robustness, CapacityBetter NCC values -Better robustness than LSB -PSNR >30 dB -Improves capacity over ISBLess Robust against geometric attacks, like scaling and rotation. -Limited to one pixelImage authentication ISB [69] Grayscale256256, 9090watermarked image qualityImproved robustness based on the NCC and PSNR values - Robust against blurring, filtering, compression, and noise.Less Robust against geometric attacks, like scaling and rotation.Image authentication Generalized patchwork [74]- - RobustnessBetter robustness against compression attacksNot robust against random bend attacksUsed for large areas of random texture image DCT and hash key [77]-512512, 6464Robustness and securityRobust against common image processing operations -SecureFragile in case of tamperingImage authenticationInformation 2020 ,11, 110 25 of 38 Table 3. Cont . Used Techniques Image TypeImage Size (pixels) (Host Image and Watermark Image Respectively)Factors Advantages Limitations Applications DCT [12] Grayscale512 x 512, 6464High capacity and robustnessCapable of embedding 4096 bits -Robust against Gaussian low pass filter and JPEG compressionLess Robust against geometric attacks, like scaling, rotation, filtering, and cropping. -Less imperceptibleImage authentication DCT and CRT [ 79] Grayscale512512, 6464 or 12864Robustness, imperceptibility, and securityLess computational complexity than SVD-Improves security -Robust to JPEG compression attacks, brightening, and sharpeninge ectsLess robust to tampering attackImage authentication DCT and linear interpolation [81]Color256256, 256256RobustnessRobust against rotational attacks, noising attacks, JPEG compression attacks, and median filtering attacksComplexIntegrity verification, tamper detection, image authentication, copyright protection DCT and repetition code [82]Color512512, 6464Robustness, imperceptibilityHigher PSNR value, -Better robustness against filtering, noising and geometric attacksHigher computational complexityCopyright ownership DCT and fractal encoding [83]Grayscale10241024, 256256RobustnessBetter robustness, -Good PSNR, -Improves securityHigher computational complexityCopyright ownership Integer DCT, non-linear chaotic map, and DSR [85]Grayscale256256, 256256Robustness, imperceptibilitySolves false positive detection problem, -Better robustness against geometric and non-geometric attacksLess robust against histogram equalization and wrappingImage authenticationInformation 2020 ,11, 110 26 of 38 Table 3. Cont . Used Techniques Image TypeImage Size (pixels) (Host Image and Watermark Image Respectively)Factors Advantages Limitations Applications DCT, Arnold transform, and chaotic encryption [24]Grayscale and color512512, 6464Robustness, imperceptibility, Payload capacityRobust against JPEG compression, rotation, cropping, Gaussian noise, filtering, and combined attacks, -Highly secureLess robust against cropping operationCopyright protection and ownership verification SCDFT and QFT [87]Color512480, -Robustness, imperceptibilityRobust against geometric transformations, Gaussian noise, and image enhancement, -Maximizes imperceptibilityNot robust against JPEG compression and color conversion, -Higher computational complexityCopy control and transaction tracking DFT [88] Bitmap512512, -Quality of watermarked imageMinimizes quality degradation of watermarked image, -Robust against amplitude modulation, PS, half-toning, PC, and attacks from the StirMark benchmark software, -Low complexityLess robust against croppingImage authentication DFT [89]DICOM Grayscale5125128 bits, -Robustness -Quality of watermarked image -Payload capacityRobust against JPEG compression, sharpening, filtering, and Gaussian noise, -Robust against geometric attacks, like rotation and scaling - Avoids the detachment problem -Better imperceptibility -Good PSNRNot capable of restoring the EPR data to their original text formatMedical image managementInformation 2020 ,11, 110 27 of 38 Table 3. Cont . Used Techniques Image TypeImage Size (pixels) (Host Image and Watermark Image Respectively)Factors Advantages Limitations Applications QDFT and log- polar transform [90]Color512384 or 384512, -Robustness, SecurityRobust against large angle rotation operations, JPEG compression, average and median filtering, and brightness adjustment, -SecuredNot robust against a type of tamperingContent authentication DFT [91] Color256256, -Robustness, Image qualityRobust against filtering, Blurring, sharpness, and Gamma noiseNot robust against geometric operationsCopyright protection and authenticity DFT and Chaotic system [92]Grayscale256256, 5050Robustness, securityRobust against JPEG compression, cropping, and noiseLess robust against rotation operation,-Complex to computeCryptology DWT [97] Grayscale256256, 3232Robustness, imperceptibilityRobust against salt-and-pepper noise, JEPG compression, rotation, and median filtering -Good imperceptibility, -PSNR =89.1481, NC =1.0000Not robust against croppingContent authentication DWT [98]Color and GrayscaleColor 512512, Grayscale 256256Robustness, image qualityRobust against Gaussian noise, salt- and-pepper noise, speckle noise, and brightnessLess robust against transformation operationCopyright protection and owner information DWT and QR Decomposition [99]Color512512, 3232Robustness, imperceptibilityRobust against compression, cropping, filtering, and noise adding, -Better imperceptibility,Less robust against salt-and-pepper noise and croppingCopyright protectionInformation 2020 ,11, 110 28 of 38 Table 3. Cont . Used Techniques Image TypeImage Size (pixels) (Host Image and Watermark Image Respectively)Factors Advantages Limitations Applications DWT and chaotic system [100]Grayscale512512, -Robustness, securitySecured against statistical attacksComplexMicrocontroller circuits’ DWT and encryption [101]Color and GrayscaleColor 228228, Grayscale 9090Robustness, imperceptibilityRobust against rotation, JPEG compression, and salt- and-pepper noise, -Better imperceptibility, -PSNR >50 dBNot robust against cropping, scaling, and other transformationsCopyright protection, content authentication DWT and Haar wavelet [102]Color256256, 6464Robustness, imperceptibilityRobust against lossy compression and Gaussian noiseComplexSecurity of image information SVD [104] Grayscale512512, 3232Robustness, image quality, securityRobust against JPEG compression, Gaussian noise, sharpening, and cropping, -Preserves image qualityNot robust against rotation and scalingOwnership identification SVD [105]Color and Grayscale256256, 256256Robustness, securityRobust against Gaussian noise, Salt-and-pepper noise, motion blur, median filtering, and JPEG compressionNot robust against rotation, cropping, and scalingDigital security of an image SVD and Redistributed image normalization [106]Grayscale512512, 6464Robustness and securitySolves false positive detection problem, -Better robustness and imperceptibilityDoes not work for color imagesOwnership identification, medical image watermarking, and fingerprinting SVD and Homomorphic Transform [107]Grayscale512512, 512512Robustness and imperceptibilityRobust against large rotation, cropping, scaling, JPEG compression, salt- and-pepper noise, Gaussian noise, andaverage filteringLow capacity of data embedding, - Major changes in singular values due to small changes in imageDigital security of an imageInformation 2020 ,11, 110 29 of 38 6. Challenges of Image Watermarking Methods At present, information is an asset. With the advent of computers, the usage of multimedia technology is increasing daily. This makes the tasks of protecting information from being accessed by unauthorized parties (confidentiality), ensuring the authenticity of information and protecting against unauthorized changes (integrity), and confirming that information is accessible by authorized users (availability) more challenging. These are the three key security requirements of a system, which are very di cult and challenging to implement. Moreover, robustness, imperceptibility, and capacity are the essential requirements in designing a robust watermarking system. However, keeping a balance among these three conflicting requirements is a di cult task. Imperceptibility can be achieved by embedding a watermark in the high-frequency components; however, this task produces weaker robustness, as robustness occurs in the low-frequency components. Still, security is a big challenge in digital image watermarking. More recently, internet of things (IoT)-based authentication schemes have provided supreme security without human interaction [ 114], where more encryption can be done outside the image contents. Furthermore, blockchain-based authentication schemes also provide high levels of security. Blockchain technology stores data in a decentralized manner and completely protects data against any tampering [ 115]. It also detects forgery and di erentiates the original image from the tampered image. Therefore, these two schemes can be accommodated in the watermark domain. 6.1. Attacks on Watermarks The extensive literature of various watermarking techniques reveals that the extraction or alteration of hidden watermark data is not such a di cult task for anyone, as information passes through the communication channel. However, an important trait is that the watermarking system should be robust enough against attacks. In a watermarking system, any processing that may cause the harmful detection of the watermark or impairment of the communication conveyed by the watermark is known as an attack. Then, the processed watermark data is identified as attacked data [ 116,117]. These attacks (which may be intentional or unintentional) cause distortions in the watermarked image, and include active attacks, passive attacks, geometric attacks, removal attacks, protocol attacks, cryptographic attacks, blind attacks, informed attacks, tampering attacks, simple attacks, attacks based on key estimation, destruction attacks, and synchronization attacks, among others [ 118]. This sub-section details some of the existing image watermarking attacks. 6.1.1. Active Attacks Active attacks occur when a hacker finds and exploits the weakness of a watermark detection function by removing or destroying the watermark; that is, simply by accessing the watermark embedding function, an adversary can easily distort the watermarked image. The most common active attacks for image watermarking include elimination, collusion, masking, distortion, forgery, copy, ambiguity, and scrambling attacks. In elimination attacks, the watermark image will never be detected, but the attacker tries to produce a similar output image, where the copy attack produces a copy with no watermark. On the other hand, with a masking attack, the attacked watermarked image still contains a watermark which is imperceptible by existing detectors. In distortion attacks, some processing techniques may be evenly applied to degrade the watermark, either over the whole watermarked image or some part of it. In a forgery attack, an invalid watermark image can be falsely authenticated by the detector for performing unauthorized embedding by an adversary. However, an ambiguity attack sometimes occurs when an adversary produces an output as forgery, even after the watermarked image is validated. A scrambling attack may be caused by detecting a valid watermarked image as a fake image [ 119]. Defenses must be carried out to protect active attacks, which are used, for example, for fingerprinting, copyright protection, and copy control.Information 2020 ,11, 110 30 of 38 6.1.2. Passive Attacks A passive attack happens when an attacker tries to find whether a given watermark is present or not without concern for the removal (destruction or deletion) of the watermark. The attacker does not try to modify the watermarking resources but, rather, to obtain the information associated to it. At this time, di erent levels of passive attacks can be considered for achieving various goals that are important in hidden communication. 6.1.3. Removal Attacks Removal attacks try to remove the watermark from the host image without using the key used in the watermark embedding. These attacks are essential, and the category includes blind watermark removal, collusion attacks, remodulation, interference attacks, noise attacks, denoising, quantization, and lossy compression, among others. These attacks cannot remove the watermark completely, but attempt to damage the watermark information considerably. The original owner attempts to make it challenging to detect the watermark due to removal attacks, as it decreases the robustness level of the watermark signal. A remodulation attack modifies the watermark image by using the modulation technique. This attack demodulates the same watermark image with the help of opposite modulation techniques. On the other hand, collusion attacks arise when hackers remove the watermark from the original data and construct a new copy without a watermark from several copies of the same original data. Each original copy contains di erent watermarking techniques. All types of noise, including Gaussian noise, additive noise, and salt-and-pepper noise, are used in noise attacks, which add a noise signal to the watermarked image, which causes the sender of the data to become confused. An interference attack may occur due to additional noise being added to the watermarked image [ 120]. These attacks attempt to harm the embedded watermark without deteriorating the document quality [121]. 6.1.4. Geometric Attacks The existing conventional watermarking algorithms are said to be e cient if they are robust against (intentional or unintentional) geometric attacks. These do not try to remove the watermark image itself but, rather, attempt to distort the watermark detector synchronization by using the inserted information. This is opposite in manner to removal attacks, and results in great di culty in the required synchronization process in recovering the embedded watermark information by the detector. Therefore, synchronization errors between the original watermark and the extracted watermark occur during the watermark extraction process. However, the watermark still exists in the watermarked image, due to changed positions. Hence, image transformation, image degradation, image enhancement, image compression, cropping, and image adjustment are all sorts of geometric attacks, as such manipulation aects the image geometry, which must be rejected to ensure the robustness of the system. Image transformation can prevent the blind detection of a public watermark by only performing rotation, scaling, and translation (RST) operations on an image to reduce the robustness level of that image. Hence, a robust watermarking system for images must be designed which is invariant to RST operations. The algorithm proposed by Lin [ 122], where an RST-invariant signal is created by taking the Fourier transform of the image and then resampling and integrating along the radial dimension. Additionally, removing some parts of the host image degrades the image quality, resulting in image degradation attacks. Degradation attacks need to be designed for some restoration methods for reducing or eliminating the degradation. Another attack processes a given image by increasing the dynamic range of the chosen features to obtain more suitable results for a specific application. These attacks can be easily detected, and this property is known as image enhancement. Image compression attack reduces the amount of data of the watermark image and cuts the bandwidth required to represent a digital image. Finally, the alteration of brightness, contrast, gamma value, and saturation result in image adjustment attacks, which change the watermark image.Information 2020 ,11, 110 31 of 38 6.1.5. Protocol Attacks Attacks that are directly aimed at the watermarking application are known as a protocol attacks. A protocol attack can be either an invertible attack, an ambiguity attack, or a copy attack. Non-invertible watermarks may be needed for copyright protection applications, where a watermark can never be extracted from a non-watermarked document. Invertible watermark attacks happen when the watermark is subtracted from the watermarked data by the attacker, who claims they are the owner of the watermarked data, which creates ambiguity about the original owner of the data [ 123,124]. This scheme results in ambiguity attacks, inversion attacks, deadlock attacks, fake-original attacks, or fake watermark attacks. A copy attack copies the watermark to some other data, called target data, after estimating it from the watermarked data without destroying the watermark or impairing the watermark detection [124]. 6.1.6. Cryptographic Attacks Cryptographic attacks may be either a security attack or an oracle attack, aimed at cracking the security in watermarking schemes by removing the embedded watermark information. A brute-force search embeds secret information which misleads the watermark. Another attack, which creates a non-watermarked signal with an available public watermark detector device, is known as an oracle attack [ 125]. Applications must restrict these types of attacks used in cryptography due to their high computational complexity. 6.2. Cost-E ectiveness of Di erent Attacking Scenarios The cost-e ectiveness of di erent attacks on digital image watermarking, which is usually based on computational complexity, indicates the cost (time and memory space) it requires to complete an attack. Watermarking is mainly involved with key and embedding algorithms, which are also important parameters for an attack. Di erent attacks are associated with di erent parameters. All cost-e ective parameters can be best described as in Table 4. Here,  K: cost of finding the key. This includes the e ective length of the key, which measures the security of the watermarking algorithm;  E: the embedding cost, which a ects the robustness and imperceptibility of the watermarking algorithm. This cost estimates the watermark embedding strength;  R: the cost to remove the watermark by an attacker from the host image without using the key used in the watermark embedding algorithm;  G: the geometric distortion cost;  E1: the new embedding cost generated by an attacker. Table 4. Cost of di erent attacks ( K: Key, E: Embedding, R: Removal, G: Geometric distortion, E1: New Embedding). Attacks Cost Active K+E+R Passive K+E Removal R Geometric K+E+G Protocol K+E+R+E1 Cryptographic K Cryptographic cost is determined by finding the key Kthrough a brute-force attack.Information 2020 ,11, 110 32 of 38 6.3. Performance Metrics for Evaluating Watermarking System Quality is an important criterion for recognizing an image-based object. The performance of watermarked image quality is measured by evaluating some performance metrics and benchmark tools, such as PSNR, MSE, Euclidean Distance (ED), SSIM, the Feature Similarity Indexing Method (FSIM), Image Fidelity (IF), Normalized Cross-Correlation (NCC), Normalized Mean Squared Error (NMSE), and Correlation Quality (CQ), among others. PSNR is expressed as the ratio of the maximum possible power of a signal to the power of corrupting noise. PSNR a ects the reliability of the system, which is best described by MSE. In statistics, the MSE calculates the average squared intensity di erences between the reference watermark and the extracted watermark. A higher PSNR value indicates a more ecient system, which means there exists no visual distinction between an ideal image and a corrupted image. The SSIM and FSIM, which are alternatives to PSNR and MSE, respectively, are used to compare the similarity measures (structures and features) between the original and recovered images, based on perception. The SSIM is used to predict the image quality of color (i.e., RGB) values or chromatic (i.e., YCbCr) values by evaluating how much an ideal image is distorted or degraded. The FSIM measures the similarities between the features of two images. One study [ 126] showed that SSIM and FSIM provide perception errors based on the human visual system, while PSNR and MSE provide absolute errors. Therefore, the performance metrics SSIM and FSIM are easy to understand for measuring performance, compared to PSNR and MSE. Furthermore, in signal processing, NCC measures the similarity between the reference watermark and the extracted watermark. NCC is defined as a function without subtracting the local mean value of intensities. These benchmark tools are commonly used to assess the performance of watermarking systems. 7. Conclusions and Future Directions At present, information can be duplicated easily due to the interactive and digital communication of multimedia data. This issue makes digital image watermarking a significant field of research. Digital image watermarking using various techniques has been applied as an important tool for image authentication, integrity verification, tamper detection, copyright protection, and the digital security of an image. In this study, we reviewed the most dominant state-of-the-art watermarking techniques. Through this study, it can be concluded that DWT is a high-quality and robust technique for image watermarking due to its multi-resolution characteristics. Robustness, imperceptibility, and capacity are the essential requirements in designing an e cient watermarking system. However, it is almost impossible to achieve all of these requirements simultaneously. Therefore, a good trade-o  between these three requirements must be maintained. However, security remains a big challenge in digital image watermarking technologies, and the accommodation of IoT and blockchain-based authentication schemes provides a challenge for researchers. Therefore, future work can be extended by combining various techniques in di erent domains to fulfill the above three important requirements. Moreover, to improve robustness along with security, researchers should focus on developing new, advanced techniques. Author Contributions: M.B. studied and drafted the whole paper; M.S.U. initiated the concept, supervised the study, and fine-tuned the manuscript. All authors have read and agreed to the published version of the manuscript. Funding: This research received no external funding. Conflicts of Interest: The authors declare no conflict of interest. References 1. Tao, H.; Chongmin, L.; Zain, J.M.; Abdalla, A.N. Robust Image Watermarking Theories and Techniques: A Review. J. Appl. Res. Technol. 2014 ,12, 122–138. [CrossRef] 2. Zhang, Y. Digital Watermarking Technology: A Review. In Proceedings of the ETP International Conference on Future Computer and Communication, Wuhan, China, 6–7 June 2009; pp. 250–252.Information 2020 ,11, 110 33 of 38 3. Cox, I.; Miller, M.; Bloom, J.; Fridrich, J.; Kalker, T. Digital Watermarking and Steganography , 2nd ed.; The Morgan Kaufmann Series in Multimedia Information and Systems: Burlington, Massachusetts, 2008. 4. Mohanarathinam, A.; Kamalraj, S.; Venkatesan, G.P .; Ravi, R.V .; Manikandababu, C.S. Digital Watermarking Techniques for Image Security: A Review. J. Ambient Intell. Humaniz. Comput. 2019 , 1–9. [CrossRef] 5. Cox, I.J.; Miller, M.L. “Review of watermarking and the importance of perceptual modeling”. Proc. SPIE 1997 ,3016 . [CrossRef] 6. Yang, Q.; Zhang, Y.; Yang, C.; Li, W. Information Entropy Used in Digital Watermarking. In Proceedings of the 2012 Symposium on Photonics and Optoelectronics, Shanghai, China, 21–23 May 2012; pp. 1–4. 7. Yu, C.; Li, X.; Chen, X.; Li, J. An Adaptive and Secure Holographic Image Watermarking Scheme. Entropy 2019 ,21, 460. [CrossRef] 8. Kumar, V .A.; Rao, C.H.S.; Dharmaraj, C. Image Digital Watermarking: A Survey. Int. J. Adv. Manag. Technol. Eng. Sci. 2018 ,8, 127–143. 9. Jaynes, E.T. Prior probabilities. IEEE Trans. Syst. Sci. Cybern. 1968 ,4, 227–241. [CrossRef] 10. Refregier, P .; Javidi, B. Optical Image Encryption based on Input Plane and Fourier Plane Random Encoding. Opt. Lett. 1995 ,20, 767–769. [CrossRef] 11. Wu, C.; Ko, J.; Rzasa, J.R.; Paulson, D.A.; Davis, C.C. Phase and Amplitude Beam Shaping with Two Deformable Mirrors Implementing Input Plane and Fourier Plane Phase Modifications. Appl. Opt. 2018 ,57, 2337–2345. [CrossRef] 12. Pun, C.M. High Capacity and Robust Digital Image Watermarking. In Proceedings of the 5th International Joint Conference on INC, IMS and IDC, Seoul, South Korea, 25–27 August 2009; pp. 1457–1461. 13. Yang, H.M.; Liang, Y.Q.; Wang, X.D.; Ji, S.J. A DWT-Based Evaluation Method of Imperceptibility of Watermark in Watermarked Color Image. In Proceedings of the 2007 International Conference on Wavelet Analysis and Pattern Recognition, Beijing, China, 2–4 November 2007; pp. 198–203. 14. Zhang, H.; Wang, C.; Zhou, X. A Robust Image Watermarking Scheme Based on SVD in the Spatial Domain. Future Internet 2017 ,9, 45. [CrossRef] 15. Takore, T.T.; Kumar, P .R.; Devi, G.L. A New Robust and Imperceptible Image Watermarking Scheme Based on Hybrid Transform and PSO. Int. J. Intell. Syst. Appl. 2018 ,11, 50–63. [CrossRef] 16. Liu, J.; He, X. A Review Study on Digital Watermarking. In Proceedings of the 1st International Conference on Information and Communication Technologies, ICICT, Karachi, Pakistan, 27–28 August 2005; pp. 337–341. 17. Olanrewaju, R.F. Development of Intelligent Digital Watermarking via Safe Region. Ph.D. Thesis, Kulliyyah of Engineering, International Islamic University Malaysia, Selangor, Malaysia, 2011. 18. Yadav, U.; Sharma, J.P .; Sharma, D.; Sharma, P .K. Di erent Watermarking Techniques & its Applications: A Review. Int. J. Sci. Eng. Res. 2014 ,5, 1288–1294. 19. Cvejic, N. Algorithms for Audio Watermarking and Steganography. Master’s Thesis, Department of Electrical and Information Engineering, University of Oulu, Oulu, Finland, 2004. 20. Zhang, H.; Wang, C.; Zhou, X. Fragile Watermarking for Image Authentication. Using the Characteristic of SVD. Algorithms 2017 ,10, 27. [CrossRef] 21. Sang, J.; Alam, M.S. Fragility and Robustness of Binary-Phase-Only-Filter-Based Fragile /Semi fragile Digital Image Watermarking. IEEE Trans. Instrum. Meas. 2008 ,57, 595–606. [CrossRef] 22. Zhang, Y.; Wang, C.; Wang, X.; Wang, M. Feature-Based Image Watermarking Algorithm Using SVD and APBT for Copyright Protection. Future Internet 2017 ,9, 13. [CrossRef] 23. Zhou, X.; Zhang, H.; Wang, C. A Robust Image Watermarking Technique Based on DWT, APDCBT, and SVD. Symmetry 2018 ,10, 77. [CrossRef] 24. Loani, N.A.; Hurrahi, N.N.; Parah, S.A.; Lee, J.W.; Sheikhi, J.A.; MohiuddinBhat, G. Secure and Robust Digital Image Watermarking Using Coe cient Di erencing and Chaotic Encryption. IEEE Access 2018 ,6, 19876–19897. [CrossRef] 25. Castiglione, A.; Pizzolante, R.; Palmieri, F.; Masucci, B.; Carpentieri, B.; De Santis, A. On-Board Format-Independent Security of Functional Magnetic Resonance Images. ACM Trans. Embed. Comput. Syst. 2017 ,16, 1–15. [CrossRef] 26. Wang, C.; Zhang, H.; Zhou, X. A Self-Recovery Fragile Image Watermarking with Variable Watermark Capacity. Appl. Sci. 2018 ,8, 548. [CrossRef] 27. Zhang, F.; Zhang, H. Digital Watermarking Capacity and Reliability. In Proceedings of the IEEE International Conference on e-Commerce Technology, San Diego, CA, USA, 9 July 2004; pp. 295–298.Information 2020 ,11, 110 34 of 38 28. Katti, S.J.; Namuduri, V .R.; Namuduri, K.R. A Practical Approach for Evaluating the Capacity of Watermarking Channel. In Proceedings of the International Conference on Intelligent Sensing and Information Processing, Chennai, India, 4–7 January 2005; pp. 193–198. 29. Kavitha, K.J.; Shan, B.P . Implementation of DWM for Medical Images using IWT and QR Code as a Watermark. In Proceedings of the IEEE Conference on Emerging Devices and Smart Systems, Tiruchengode, India, 3–4 March 2017; pp. 252–255. 30. Pizzolante, R.; Castiglione, A.; Carpentieri, B. Protection of Microscopy Images through Digital Watermarking Techniques. In Proceedings of the International Conference on Intelligent Networking and Collaborative Systems, Salerno, Italy, 10–12 September 2014; pp. 65–72. 31. Ling, H.-C.; Phan, R.C.-W.; Heng, S.-H. Comment on Robust Blind Image Watermarking Scheme Based on Redundant Discrete Wavelet Transform and Singular Value Decomposition. AEU-Int. J. Electron. Commun. 2013 ,60, 894–897. [CrossRef] 32. Goos, G.; Hartmanis, J.; Van Leeuwen, J. Cloud Computing and Security. In Proceedings of the 4th International Conference, ICCCS, Haikou, China, 8–10 June 2018; pp. 691–697. 33. P’erez-Freire, L.; Na, P .C.; Ramon, J.; Troncoso-Pastoriza, J.R.; Gonzalez, F.P . Watermarking Security: A Survey. InTransactions on Data Hiding and Multimedia Security ; Lecture Notes in Computer Science: Berlin /Heidelberg, Germany, 2006; pp. 41–72. 34. Bruce, A.M. A Review of Digital Watermarking. Available online: https: //pdfs.semanticscholar.org /d6eb/ c1a3e1676df1b5a32033417215e8da096ac4.pdf (accessed on 16 February 2020). 35. Chopra, J.; Kumar, A.; Kumar, A.; Marwaha, A. An E cient Watermarking for Protecting Signature Biometric Template. In Proceedings of the 5th International Conference on Signal Processing and Integrated Networks (SPIN), Noida, India, 22–23 February 2018; pp. 413–418. 36. Singh, A.K.; Kumar, B.; Singh, G.; Mohan, A. Medical Image Watermarking. Multimedia Systems and Applications ; Springer: Berlin /Heidelberg, Germany, 2017. 37. Qasim, A.F.; Meziane, F.; Aspin, R. Digital watermarking: Applicability for Developing Trust in Medical Imaging Workflows State of the Art Review. Comput. Sci. Rev. 2018 ,27, 45–60. [CrossRef] 38. De Vleeschouwer, C.; Delaigle, J.; Macq, B. Invisibility and Application Functionalities in Perceptual Watermarking-An Overview. Proc. IEEE 2002 ,90, 64–77. [CrossRef] 39. Merrad, A. Implementation of a Biometric Speech Watermarking Based on Wavelet Transform. Ph.D. Thesis, Ziane Achour University of Djelfa, Djefa, Algeria, 2019. 40. Singh, A.K.; Kumar, B.; Singh, S.K.; Ghrera, S.P .; Mohan, A. Multiple Watermarking Technique for Securing Online Social Network Contents Using Back Propagation Neural Network. Future Gener. Comput. Syst. 2016 , 86, 926–939. [CrossRef] 41. Zear, A.; Singh, A.K.; Kumar, P . A Proposed Secure Multiple Watermarking Technique Based on DWT, DCT and SVD for Application in Medicine. Multimed Tools Appl. 2016 ,77, 4863–4882. [CrossRef] 42. Phadikar, A.; Jana, P .; Mandal, H. Reversible Data Hiding for DICOM Image Using Lifting and Companding. Cryptography 2019 ,3, 21. [CrossRef] 43. Yusof, Y.; Khalifa, O.O. Digital Watermarking for Digital Images using Wavelet Transform. In Proceedings of the IEEE International Conference on Telecommunications and Malaysia International Conference on Communications, Penang, Malaysia, 14–17 May 2007; pp. 665–669. 44. Singh, V . Digital Watermarking: A Tutorial. Available online: http: //www.cyberjournals.com /Papers /Jan2011 / 02.pdf (accessed on 16 February 2020). 45. Agbaje, M.; Olugbenga Awodele, O.; Idowu, S.A. Broadcast Monitoring and Applications. J. Telecommun. 2012 ,7, 11–16. 46. Kaur, E.J.; Kaur, E.K. Digital Watermark: A Study. Int. J. Adv. Res. Comput. Sci. Softw. Eng. 2012 ,2, 159–163. 47. Hsu, C.S.; Tu, S.F. Digital Watermarking Scheme for Copyright Protection and Tampering Detection. Int. J. Inf. Technol. Secur. 2019 ,11, 107–119. 48. Hamidi, M.; Chetouani, A.; El Haziti, M.; El Hassouni, M.; Cherifi, H. Blind Robust 3D Mesh Watermarking Based on Mesh Saliency and Wavelet Transform for Copyright Protection. Information 2019 ,10, 67. [CrossRef] 49. Kunhu, A.; Al Mansoori, S.; Al-Ahmad, H. A Novel Reversible Watermarking Scheme Based on SHA3 for Copyright Protection and Integrity of Satellite Imagery. Int. J. Comput. Sci. Netw. Secur. 2019 ,19, 92–102. 50. Furon, T. A Survey of Watermarking Security ; International Workshop on Digital Watermarking: Siena, Italy, 2005; pp. 201–215.Information 2020 ,11, 110 35 of 38 51. Rashid, A. Digital Watermarking Applications and Techniques: A Brief Review. Int. J. Comput. Appl. Technol. Res. 2016 ,5, 147–150. 52. Adnan, W.A.W.; Hitarn, S.; Abdul-Karim, S.; Tamjis, M.R. A Review of Image Watermarking. In Proceedings of the Student Conference on Research and Development, Putrajaya, Malaysia, 25–26 August 2003; pp. 381–384. 53. Mahajan, J.R.; Patil, N.N. Alpha Channel for Integrity Verification using Digital Signature on Reversible Watermarking QR. In Proceedings of the 2015 International Conference on Computing Communication Control and Automation, Pune, India, 26–27 February 2015; pp. 602–606. 54. Tohidi, F.; Paul, M.; Hooshmandasl, M.R.; Debnath, T.; Jamshidi, H. E cient Self-embedding Data Hiding for Image Integrity Verification with Pixel-Wise Recovery Capability. In Pacific-Rim Symposium on Image and Video Technology ; Springer: Cham, Switzerland, 2019; Volume 11854, pp. 128–141. 55. Allaf, A.H.; Kbir, M.A.; Allaf, A.H.; Kbir, M.A. A Review of Digital Watermarking Applications for Medical Image Exchange Security. In The Proceedings of the Third International Conference on Smart City Applications ; Springer: Cham, Switzerland, 2019; pp. 472–480. 56. Wu, N.I.; Hwang, M.S. Data Hiding: Current Status and Key Issues. Int. J. Netw. Secur. 2007 ,4, 1–9. 57. Celik, M.; Sharma, G.; Saber, E.; Tekalp, A. Hierarchical Watermarking for Secure Image Authentication with Localization. IEEE Trans. Image Process. 2002 ,11, 585–595. [CrossRef] 58. Mukherjee, D.P . Spatial Domain Digital Watermarking of Multimedia Objects for Buyer Authentication. IEEE Trans. Multimed. 2004 ,6, 1–15. [CrossRef] 59. Nikolaidis, N.; Pitas, I. Robust Image Watermarking in the Spatial Domain. Signal Process. 1998 ,66, 385–403. [CrossRef] 60. Habes, A. Information Hiding in BMP Image Implementation, Analysis and Evaluation. Inf. Transm. Comput. Netw. 2006 ,6, 1–10. 61. Abdullatif, M.; Zeki, A.M.; Chebil, J.; Gunawan, T.S. Properties of Digital Image Watermarking. In Proceedings of the IEEE 9th International Colloquium on Signal Processing and its Applications, Kuala Lumpur, Malaysia, 8–10 March 2013; pp. 235–240. 62. Fung, A.G.C.; Junior, W.G. A Review Study on Image Digital Watermarking. In Proceedings of the 10th International Conference on Networks, St Maarten, The Netherlands, 22–23 January 2011; pp. 24–28. 63. Manjula, G.R.; Danti, A. A Novel Hash Based Least Significant Bit (2-3-3) Image Steganography in Spatial Domain. Int. J. Secur. Priv. Trust Manag. 2015 ,4, 11–20. 64. Abraham, J.; Paul, V . An Imperceptible Spatial Domain Color Image Watermarking Scheme. J. King Saud Univ. 2019 ,31, 125–133. [CrossRef] 65. Muyco, S.D.; Hernandez, A.A. Least Significant Bit Hash Algorithm for Digital Image Watermarking Authentication. In Proceedings of the 5th International Conference on Computing and Artificial Intelligence, Bali, Indonesia, 19–22 April 2019; pp. 150–154. 66. Zeki, A.M.; Manaf, A.A. A Novel Digital Watermarking Technique Based on ISB (Intermediate Significant Bit). World Acad. Sci. Eng. Technol. Int. J. Comput. Inf. Eng. 2009 ,3, 444–451. 67. Mohammed, G.N.; Yasin, A.; Zeki, A.M. Robust Image Watermarking Based on Dual Intermediate Significant Bit (DISB). In Proceedings of the 6th International Conference on CSIT, Amman, Jordan, 26–27 March 2014; pp. 19–22. 68. Jane, O.; Elbasi, E. A New Approach in Non-blind Watermarking method Based on DWT and SVD via LU Decomposition. Turk. J. Electr. Eng. Comput. Sci. 2014 ,22, 1354–1366. [CrossRef] 69. Zeki, A.; Abubakar, A.; Chiroma, H. An Intermediate Significant Bit (ISB) Watermarking Technique Using Neural Networks. Available online: https: //link.springer.com /article /10.1186 /s40064-016-2371-6#citeas (accessed on 16 February 2020). 70. Rathor, B.; Saharan, R. Steganography using Bit Plane Embedding and Cryptography. In Proceedings of the 1st International Conference on Smart System, Innovations and Computing, Jaipur, India, 15–16 April 2017; Volume 79, pp. 319–330. 71. Bender, W.; Gruhl, D.; Morimoto, N.; Lu, A. Techniques for Data Hiding. IBM Syst. J. 1996 ,35, 313–336. [CrossRef] 72. Singh, P .; Chadha, R.S. A Survey of Digital Watermarking Techniques, Applications and Attacks. Int. J. Eng. Innov. Technol. 2013 ,2, 165–175.Information 2020 ,11, 110 36 of 38 73. Wu, X.; Hu, J.; Gu, Z.; Huang, J. A Secure Semi-fragile Watermarking for Image Authentication Based on Integer Wavelet Transform with Parameters. Available online: https: //dl.acm.org /doi/10.5555 /1082290.1082302 (accessed on 16 February 2020). 74. Yeo, I.N.; Kim, H.J. Generalized Patchwork Algorithm for Image Watermarking. Multimed. Syst. 2003 ,9, 261–265. [CrossRef] 75. Saqib, M.; Naaz, S. Spatial and Frequency Domain Digital Image Watermarking Techniques for Copyright Protection. Int. J. Eng. Sci. Technol. 2017 ,9, 691–699. 76. Meyer-Baese, A.; Schmid, V . Feature Selection and Extraction. In Proceedings of the Pattern Recognition and Signal Analysis in Medical Imaging, Nasreen, Shamila, 27 August 2014; pp. 21–69. 77. Kitanovski, V .; Taskovski, D.; Bogdanova, S. Watermark Generation using Image-Dependent Key for Image Authentication. In Proceedings of the International Conference on “Computer as a Tool”, Belgrade, Serbia, 21–24 November 2005; pp. 947–950. 78. Chen, B.; Wornell, G.W. Quantization Index Modulation: A Class of Provably Good Methods for Digital Watermarking and Information Embedding. IEEE Trans. Inf. Theory 2001 ,47, 1423–1443. [CrossRef] 79. Patra, J.C.; Phua, J.E.; Rajan, D. DCT Domain Watermarking scheme Using Chinese Remainder Theorem for Image Authentication. In Proceedings of the International Conference on Multimedia and Expo, Suntec, Singapore, 19–23 July 2010; pp. 111–116. 80. Xu, Z.J.; Wang, Z.Z.; Lu, Q. Research on Image Watermarking Algorithm Based on DCT. Procedia Environ. Sci. 2011 ,10, 1129–1135. [CrossRef] 81. Laouamer, L.; Tayan, O. A Semi-Blind Robust DCT Watermarking Approach for Sensitive Text Images. Arab. J. Sci. Eng. 2015 ,40, 1097–1109. [CrossRef] 82. Roy, S.; Pal, A.K. A Blind DCT Based Color Watermarking Algorithm for Embedding Multiple Watermarks. AEU-Int. J. Electron. Commun. 2017 ,72, 149–161. [CrossRef] 83. Liu, S.; Pan, Z.; Song, H. Digital Image Watermarking Method Based on DCT and Fractal Encoding. IET Image Process 2017 ,11, 815–821. [CrossRef] 84. Vishwakarma, V .P .; Sisaudia, V . Gray-scale Image Watermarking Based on DE-KELM in DCT Domain. Procedia Comput. Sci. 2018 ,132, 1012–1020. [CrossRef] 85. Singh, S.P .; Bhatnagar, G. A New Robust Watermarking System in Integer DCT Domain. J. Vis. Commun. Image Represent. 2018 ,53, 86–101. [CrossRef] 86. Discrete Fourier Transform. Available online: https: //en.wikipedia.org /wiki/Discrete_Fourier_transform (accessed on 23 October 2019). 87. Tsui, T.K.; Zhang, X.; Androutsos, D. Color Image Watermarking Using Multidimensional Fourier Transforms. IEEE Trans. Inf. Forensics Secur. 2008 ,3, 16–28. [CrossRef] 88. Poljicak, A.; Mandic, L.; Agic, D. Discrete Fourier Transform Based Watermarking Method with an Optimal Implementation Radius. J. Electron. Imaging 2011 ,20, 033008. [CrossRef] 89. Cedillo-Hernandez, M.; Garcia-Ugalde, F.; Nakano-Miyatake, M.; Perez-Meana, H. Robust Watermarking method in DFT Domain for E ective Management of Medical Imaging. J. Signal Image Video Process. 2013 ,9, 1163–1178. [CrossRef] 90. Ouyang, J.; Coatrieux, G.; Shu, H. Robust Hashing for Image Authentication Using Quaternion Discrete Fourier Transform and Log-Polar Transform. J. Digit. Signal Process. 2015 ,41, 98–109. [CrossRef] 91. Gaata, M.T. An E cient Image Watermarking Approach Based on Fourier Transform. International J. Comput. Appl. 2016 ,136, 8–11. 92. Jamal, S.S.; Khan, M.U.; Shah, T. A Watermarking Technique with Chaotic Fractional S-box Transformation. J. Wirel. Peers Commun. 2016 ,90, 2033–2049. [CrossRef] 93. Raut, S.S.; Mune, A.R. A Review Paper on Digital Watermarking Techniques. Int. J. Eng. Sci. Comput. 2017 , 7, 10460–10463. 94. Discrete Wavelet Transform. Available online: https: //en.wikipedia.org /wiki/Discrete_wavelet_transform (accessed on 23 October 2019). 95. Kehtarnavaz, N. Digital Signal Processing System Design , 2nd ed.; Elsevier: Cambridge, MA, USA, 2008. 96. Najafi, E. A Robust Embedding and Blind Extraction of Image Watermarking Based on Discrete Wavelet Transform. J. Math. Sci. 2017 ,1, 307–318. [CrossRef]Information 2020 ,11, 110 37 of 38 97. Chen, Z.; Chen, Y.; Hu, W.; Qian, D. Wavelet Domain Digital Watermarking Algorithm Based on Threshold Classification. In International Conference in Swarm Intelligence ; Springer: Cham, Switzerland, 2015; Volume 9142, pp. 129–136. 98. Haribabu, M.; Bindu, C.H.; Swamy, K.V . A Secure & Invisible Image Watermarking Scheme Based on Wavelet Transform in HSI color space. In Proceedings of the 6th International Conference on Advances in Computing & Communications, Cochin, India, 6–8 September 2016; pp. 462–468. 99. Jia, S.; Zhou, Q.; Zhou, H. A Novel Color Image Watermarking Scheme Based on DWT and QR Decomposition. J. Appl. Sci. Eng. 2017 ,20, 193–200. 100. Hannoun, K.; Hamiche, H.; Lahdir, M.; Laghrouche, M.; Kassim, S. A Novel DWT Domain Watermarking Scheme Based on a Discrete-Time Chaotic System. IFAC-Pap. Line 2018 ,51, 50–55. [CrossRef] 101. Ambadekar, S.P .; Jain, J.; Khanapuri, J. Digital Image Watermarking through Encryption and DWT for Copyright Protection. J. Recent Trends Signal Image Process. 2018 ,727, 187–195. 102. Wang, J.; Du, Z. A Method of Processing Color Image Watermarking Based on the Haar Wavelet. J. Vis. Commun. Image Represent. 2019 ,64, 1–8. [CrossRef] 103. Singular Value Decomposition. Available online: https: //en.wikipedia.org /wiki/Singular_value_ decomposition (accessed on 24 October 2019). 104. Chang, C.-C.; Tsai, P .; Lin, C.-C. SVD-Based Digital Image Watermarking Scheme. J. Pattern Recognit. Lett. 2005 ,26, 1577–1586. [CrossRef] 105. Vaishnavia, D.; Subashini, T.S. Robust and Invisible Image Watermarking in RGB Color Space Using SVD. In Proceedings of the International Conference on Information and Communication Technologies, Kochi India, 3–5 December 2014; pp. 1770–1777. 106. Ali, M.; Ahn, C.W.; Pant, M.; Siarry, P . A Reliable Image Watermarking Scheme Based on Redistributed Image Normalization and SVD. Discret. Dyn. Nat. Soc. 2016 , 1–15. [CrossRef] 107. Verma, D.; Aggarwal, A.K.; Agarwal, H. Watermarking Scheme Based on Singular Value Decomposition and Homomorphic Transform. Available online: https: //aip.scitation.org /doi/abs/10.1063 /1.5008715 (accessed on 12 February 2020). 108. Shih, F.Y.; Wu, S.Y. Combinational Image Watermarking in the Spatial and Frequency Domains. J. Pattern Recognit. Soc. 2003 ,36, 969–975. [CrossRef] 109. Sridhar, P . A Robust Digital Image Watermarking in Hybrid Frequency Domain. Int. J. Eng. Technol. 2018 ,7, 243–248. [CrossRef] 110. Kumar, A. A Review on Implementation of Digital Image Watermarking Using LSB and DWT. Inf. Commun. Technol. Sustain. Dev. 2019 ,933, 595–602. 111. Abdulrahman, A.K.; Ozturk, S. A Novel Hybrid DCT and DWT Based Robust Watermarking Algorithm for Color Images. Multimed. Tools Appl. 2019 ,78, 17027–17049. [CrossRef] 112. Savakar, D.G.; Ghuli, A. Robust Invisible Digital Image Watermarking using Hybrid Scheme. Arab. J. Sci. Eng. 2019 ,44, 3995–4008. [CrossRef] 113. Liu, J.; Li, J.; Ma, J.; Sadiq, N.; Bhatti, U.A.; Ai, Y. A Robust Multi-Watermarking Algorithm for Medical Images Based on DTCWT-DCT and Henon Map. Appl. Sci. 2019 ,9, 700. [CrossRef] 114. El-hajj, M.; Fadlallah, A.; Chamoun, M.; Serhrouchni, A. A Survey of Internet of Things (IoT) Authentication Schemes. Sensors 2019 ,19, 1141. [CrossRef] 115. Dobre, R.A.; Preda, R.O.; Oprea, C.C.; Pirnog, I. Authentication of JPEG Images on the Blockchain. In Proceedings of the International Conference on Control, Artificial Intelligence, Robotics & Optimization (ICCAIRO), Prague, Czech Republic, 19–21 May 2018; pp. 211–215. 116. Yongliang, L.; Gao, W. Secure Watermark Verification Scheme. In Proceedings of the International Conference on Multimedia and Expo (ICME), Taipei, Taiwan, 27–30 June 2004; pp. 923–926. 117. Huang, C.; Wu, J. Attacking Visible Watermarking Schemes. IEEE Trans. Multimed. 2004 ,6, 16–30. [CrossRef] 118. Agarwal, N.; Singh, A.K.; Singh, P .K. Survey of Robust and Imperceptible Watermarking. Multimed. Tools Appl. 2019 ,78, 8603–8633. [CrossRef] 119. Nyeem, H.; Boles, W.; Boyd, C. Digital Image Watermarking: Its Formal Model, Fundamental Properties and Possible Attacks. EURASIP J. Adv. Signal Process 2014 ,135, 1–22. [CrossRef] 120. Chitra, K.; Venkatesan, V .P . Spatial Domain Watermarking Technique: An Introspective Study. In Proceedings of the International Conference on Informatics and Analytics, Pondicherry, India, 25–26 August 2016; pp. 1–6.Information 2020 ,11, 110 38 of 38 121. Varshney, Y. Attacks on Digital Watermarks: Classification, Implications, Benchmarks. Int. J. Emerg. Technol. 2017 ,8, 229–235. 122. Lin, C.; Wu, M.; Bloom, J.A.; Cox, I.J.; Miller, M.L.; Lui, Y.M. Rotation, Scale, and Translation Resilient Watermarking for Images. IEEE Trans. Image Process. 2001 ,10, 767–782. [CrossRef] 123. Dittmann, J.; Wohlmacher, P .; Nahrstedt, K. Using Cryptographic and Watermarking Algorithms. IEEE Multimed. 2001 ,8, 54–65. [CrossRef] 124. Kutter, M.; Voloshynovskiy, S.V .; Herrigel, A. Watermark Copy Attack. Available online: http: //www. alpvision.com /pdf/ei2000_ol.pdf (accessed on 16 February 2020). 125. Soman, K.P .; Ramachandran, K.I. Insight into Wavelets, from Theory to Practice , 3rd ed.; PHI Learning: Delhi, India, 2010. 126. Sara, U.; Akter, M.; Uddin, M.S. Image Quality Assessment through FSIM, SSIM, MSE, and PSNR-A Comparative Study. J. Comput. Commun. 2019 ,7, 8–18. [CrossRef] ©2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http: //creativecommons.org /licenses /by/4.0/).

- Paper ID: 1275d063dfae9ea716eaad5b027555501ff4f2a0 Title: Analysis of Digital Image Watermarking Techniques through Hybrid Methods A... - Paper ID: 1275d063dfae9ea716eaad5b027555501ff4f2a0 Title: Analysis of Digital Image Watermarking Techniques through Hybrid Methods Abstract: Digital image watermarking is an attractive research area since it protects the multimedia data from unauthorized access. For designing an efficient and robust digital image watermarking system, the trade-off among imperceptibility, robustness, capacity, and security must be maintained. Various studies regarding this concern have been performed to ensure these requirements by hybridizing different domains, such as spatial and transform domains. In this paper, we have presented an analytical study of the existing hybrid digital image watermarking methods. At first, we have given a standard framework for designing a hybrid method that ensures the basic design requirements of watermarking for various applications. After a brief literature review, we compared and analyzed the complexity of several existing hybrid methods in a tabular form. The... - Paper ID: 1275d063dfae9ea716eaad5b027555501ff4f2a0 Title: Analysis of Digital Image Watermarking Techniques through Hybrid Methods Abstract: Digital image watermarking is an attractive research area since it protects the multimedia data from unauthorized access. For designing an efficient and robust digital image watermarking system, the trade-off among imperceptibility, robustness, capacity, and security must be maintained. Various studies regarding this concern have been performed to ensure these requirements by hybridizing different domains, such as spatial and transform domains. In this paper, we have presented an analytical study of the existing hybrid digital image watermarking methods. At first, we have given a standard framework for designing a hybrid method that ensures the basic design requirements of watermarking for various applications. After a brief literature review, we compared and analyzed the complexity of several existing hybrid methods in a tabular form. The limitations and applications of these methods are also highlighted. Finally, we summarized the challenges of the existing methods and concluded the study by giving future research directions. Citation Count: 24 - Paper ID: 8a201501eb1582406926ae6ccf892f8bfd2dacdb Title: On the Properties of Non-Media Digital Watermarking: A Review of State of the Art Techniques Abstract: Over the last 25 years, there has been much work on multimedia digital watermarking. In this domain, the primary limitation to watermark strength has been in its visibility. For multimedia watermarks, invisibility is defined in human terms (that is, in terms of human sensory limitations). In this paper, we review recent developments in the non-media applications of data watermarking, which have emerged over the last decade as an exciting new sub-domain. Since by definition, the intended receiver should be able to detect the watermark, we have to redefine invisibility in an acceptable way that is often application-specific and thus cannot be easily generalized. In particular, this is true when the data is not intended to be directly consumed by humans. For example, a loose definition of robustness might be in terms of the resilience of a watermark against normal host data operations, and of invisibility as resilience of the data interpretation against change introduced by the watermark. In this paper, we classify the data in terms of data mining rules on complex types of data such as time-series, symbolic sequences, data streams, and so forth. We emphasize the challenges involved in non-media watermarking in terms of common watermarking properties, including invisibility, capacity, robustness, and security. With the aid of a few examples of watermarking applications, we demonstrate these distinctions and we look at the latest research in this regard to make our argument clear and more meaningful. As the last aim, we look at the new challenges of digital watermarking that have arisen with the evolution of big data. Citation Count: 71 - Paper ID: 58afa7b97c80696522e05a532bc51b0c3a73f0ab Title: An effective embedding algorithm for blind image watermarking technique based on Hessenberg decomposition Abstract: None Citation Count: 1 - Paper ID: 1703dadbfb02673bef2dfde35ae83b364c2be27d Title: Recent Digital Watermarking Approaches, Protecting Multimedia Data Ownership Abstract: As the internet networks are proliferating and diversifying across the globe, the accessibility of digital multimedia contents such as images, audios and videos become more frequent. Digital watermarking approaches ensure data authentication, ownership protection and security of digital data. This paper includes of assorted techniques of embedding and extracting watermarks, applied in time-domain/spatial-domain and transform-domain of transmission signal. It imbibes all the ideas of digital watermarking. It starts with the overview, classification, application, possible attacks, limitations, performance analysis and comparative study among various watermarking techniques. Genetic Algorithm approaches are also our primary concern so as to ameliorate each the fidelity and robustness of multimedia data. Audio watermarking is our vital survey area. Citation Count: 4 - Paper ID: 5165bdf96b0e949aea5acf0444e6c3dad6ec0a64 Title: Dual Watermarking for Color Images: A New Image Copyright Protection Model based on the Fusion of Successive and Segmented Watermarking Abstract: None Citation Count: 22 - Paper ID: 3355c3ba11695004b2b31fc445ba614536b5bd32 Title: Illustration watermarking: an object-based approach for digital images Abstract: For most applications common watermarking techniques usually spread the data to embed over the entire media, since distributing the watermark information promises an improvement in regard to security aspects, data hiding capacity or rather robustness in terms of redundancy. Distribution is controlled by a a visual or psychoacoustical model that takes limitations of the Human Visual System (HVS) and syntactical information about the signal characteristics into account. Therefore in most cases syntactical and not semantical aspects determine embedding. In our paper we introduce an approach for object based annotation watermarking which respects semantical characteristics of digital images, referred to as model for illustration watermarking. By applying a user-assisted segmentation process regions, representing semantical objects within the image, are identified and prepared for embedding. Providing robustness to typical image processing operations like cropping, scaling, compression and rotation, the proposed technique is applicable for binding additional illustrative information to selected objects within the medium. Moreover we identify the requirements of object based watermarking in consideration of imperceptibility, as well as watermark payload, and present first experimental test results. Citation Count: 12 - Paper ID: 2eb0dfddc2c1447b9fdee9b47bc8621b67e77161 Title: A robust software watermarking framework using shellcode Abstract: None Citation Count: 1 - Paper ID: be320404621a9fe7451ad6cce58094a2f1b4be3c Title: Techniques and validation for protection of embedded processors Abstract: Advances in technology scaling and miniaturization of on-chip structures have caused an increasing complexity of modern devices. Due to immense time-to-market pressures, the reusability of intellectual property (IP) sub-systems has become a necessity. With the resulting high risks involved with such a methodology, securing IP has become a major concern. Despite a number of proposed IP protection (IPP) techniques being available, securing an IP in the register transfer level (RTL) is not a trivial task, with many of the techniques presenting a number of shortfalls or design limitations. The most prominent and the least invasive solution is the integration of a digital watermark into an existing IP. In this thesis new techniques are proposed to address the implementation difficulties in constrained embedded IP processor cores. This thesis establishes the parameters of sequences used for digital watermarking and the tradeoffs between the hardware implementation cost, detection performance and robustness against IP tampering. A new parametric approach is proposed which can be implemented with any watermarking sequence. MATLAB simulations and experimental results of two fabricated silicon ASICs with a watermark circuit embedded in an ARMR Cortex R-M0 IP core and an ARMR Cortex R-A5 IP core demonstrate the tradeoffs between various sequences based on the final design application. The thesis further focuses on minimization of hardware costs of a watermark circuit implementation. A new clock-modulation based technique is proposed and reuses the existing circuit of an IP core to generate a watermark signature. Power estimation and experimental results demonstrate a significant area and power overhead reduction, when compared with the existing techniques. To further minimize the costs of a watermark implementation, a new technique is proposed which allows a non-deterministic and sporadic generation of a watermark signature. The watermark was embedded in an ARMR Cortex R-A5 IP core and was fabricated in silicon. Experimental silicon results have validated the proposed technique and have demonstrated the negligible hardware implementation costs of an embedded watermark. Citation Count: 0 - Paper ID: 42aae882290d8397b090834d9dac5e26c1281c9e Title: A Novel Watermark Embedding Scheme using Compressive Sensing inWavelet Domain Abstract: Most existing research achievements of digital watermarking techniques are in transform domain. In compari- son with spatial domain, its advantages are larger data volume, higher security and stronger robustness. But its limitations are also obvious: complex computing requirement, weak in resisting attack and anti-extraction. In this paper, a novel blind digital watermarking algorithm is proposed, which performs digital watermark embedding process in Compressive Sens- ing (CS) domain based on the characteristics of CS and Human Visual System (HVS). The sub-blocks with larger capac- ity are selected to embed the scrambled digital watermark, considering the non-uniformity of blocks. Besides that, suitable quantization steps are chose adaptively by using quantization method. Experimental results show that the algorithm ob- tains robust and invisible embedded watermark with larger capacity of data. At the same time, the ability of defending against attack or extraction of embedded watermark is greatly improved. Most important feature in our algorithm is that the watermark can be extracted without any reference to the original image. As a result, the cost of storing carrier data can be saved remarkably. Citation Count: 6 - Paper ID: 36532b3b338cfc64849fe39f4deb10b2e642fc30 Title: Robust Watermarking Techniques for the Authentication and Copyright Protection of Digital Images: A Survey Abstract: Digital image authentication and security are crucial concerns for the digital revolution since any image may be readily tampered with. Digital watermarking schemes were since been utilized to tackle a range of problems over time and involving the verification of digital images and copyright protection. In a real-world setting, watermarked contents are frequently subject to a succession of assaults which makes it necessary to strike a balance between resilience and imperceptibility in the face of attacks. There are numerous hazards to this, and numerous watermarking techniques have been created to counter them. Data with less perceptual distortion should be able to be included in a real watermarking technique in addition to restoring the original cover content. As a result, robustness, imperceptibility, and security are three crucial criteria for digital content'sauthentication and copyrightpreservation. This study therefore presents a careful review of some cutting-edge watermarking technologies used for copyright protection and authentication so as to identify their strengths and limitations. For clarity, this study shall explain some basic concepts of digital watermarking as well as the various forms of attacks on watermarks. The contributions from this work will be useful to researchers aim at developing efficient watermarking techniques Citation Count: 2

There was an error reading the PDF. Please try a different paper. There was an error reading the PDF. Please try a different paper. There was an error reading the PDF. Please try a different paper.

There was an error reading the PDF. Please try a different paper. There was an error reading the PDF. Please try a different paper. There was an error reading the PDF. Please try a different paper.

This paper does not have an open access PDF. This paper does not have an open access PDF. This paper does not have an open access PDF.

Send Orders for Reprints to reprints@benthamscience.ae The Open Cybernetics & Systemics Journal, 2015, 9, 1-6 1 1874-110X/15 2015 Be... Send Orders for Reprints to reprints@benthamscience.ae The Open Cybernetics & Systemics Journal, 2015, 9, 1-6 1 1874-110X/15 2015 Bentham Open Open Access A Novel Watermark Embedding Scheme using Comp ressive Sensing in Wav elet Domain Bin Liao* and Jintao Lv School of Electrical and Electronic Engineering, North China Electric Power University, Beijing 102206, China Abstract: Most existing research achievements of digital watermarking techniques are in transform domain. In compari-son with spatial domain, its advantages are larger data volume, higher security and stronger robustness. But its limitations are also obvious: complex computing requirement, weak in resisting attack and anti-extraction. In this paper, a novel blind digital watermarking algorithm is proposed, which performs digital watermark embedding process in Compressive Sens-ing (CS) domain based on the characteristics of CS and Human Visual System (HVS). The sub-blocks with larger capac-ity are selected to embed ... Send Orders for Reprints to reprints@benthamscience.ae The Open Cybernetics & Systemics Journal, 2015, 9, 1-6 1 1874-110X/15 2015 Bentham Open Open Access A Novel Watermark Embedding Scheme using Comp ressive Sensing in Wav elet Domain Bin Liao* and Jintao Lv School of Electrical and Electronic Engineering, North China Electric Power University, Beijing 102206, China Abstract: Most existing research achievements of digital watermarking techniques are in transform domain. In compari-son with spatial domain, its advantages are larger data volume, higher security and stronger robustness. But its limitations are also obvious: complex computing requirement, weak in resisting attack and anti-extraction. In this paper, a novel blind digital watermarking algorithm is proposed, which performs digital watermark embedding process in Compressive Sens-ing (CS) domain based on the characteristics of CS and Human Visual System (HVS). The sub-blocks with larger capac-ity are selected to embed the scrambled digital watermark, considering the non-uniformity of blocks. Besides that, suitable quantization steps are chose adaptively by using quantization method. Experimental results show that the algorithm ob-tains robust and invisible embedded watermark with larger capacity of data. At the same time, the ability of defending against attack or extraction of embedded watermark is greatly improved. Most important feature in our algorithm is that the watermark can be extracted without any reference to the original image. As a result, the cost of storing carrier data can be saved remarkably. Keywords: Blind extraction, compressive sensing (CS), digital watermark. 1. INTRODUCTION Digital watermarking technology has many applications in different fields. Especially, there are broad application prospects in copy protection and content authentication. With the rapid development of advanced digital multimedia compression, network communication and information proc-essing technology, the multimedia information on the Inter-net expands rapidly. More and more images are being readily available both to professional and amateur users due to as-tonishing advancements in imaging technologies. Mean-while, the research in information security attracts more and more attention. Previous researches were mainly focused on spatial domain, for example, Least Significant Bit [1] (LSB) or transform domain like Discrete Wavelet Transform (DWT). Spatial domain watermarking algorithm needs sim-ple operation but has weak robustness. And in transform domain the opposite is true. In this paper, the theory of Compressive Sensing [2] (CS) is employed for digital wa-termarking technology. Blind digital watermark embedding process is performed in CS domain based on the characteris-tics of CS and HVS. This can greatly enhance the robustness and invisibility with less costs of storage space. And the abil-ity of defending against attack and extraction of embedded watermark is also greatly enhanced. CS is a hot topic in the area of information science and it is becoming “A Big Idea” in signal processing. The theory breaks the limitations of traditional sampling theory due to the synchronization of sampling and compression. CS is a *Address correspondence to this author at the School of Electrical and Elec-tronic Engineering, North China Electric Power University, Beijing 102206, China; E-mail: nathan@ncepu.edu.cn new sampling theory with extensive application prospects, which have an absolute advantage in digital watermark over DWT and DCT. In the past years, lots of researchers proposed different digital watermark algorithms with strong robustness and security. Huang [3] et al., applied CS method in embedding watermark for the first time, affirming the feasibility of CS in information hiding. Lin proposed an improved reconstruc-tion algorithm [4] according to the advantages of CS for em-bedding watermark. She picked out some special sub-blocks for sensing and got a set of coefficients for embedding addi-tive watermark in CS domain. But there is a major drawback that extracting watermark needs original image thus may cause the waste of storage. Wei [5] and others got wavelet coefficients after the DWT of carrier image. Then they used CS for high and low frequency coefficients respectively and embedded the watermark in the former. The method ensured the invisibility and robustness of watermark, but it ignored that if the same intensity of watermark is embedded in all high frequency, the robustness cannot be ensured as much as possible actually, considering that visual capacity of each block is different. What’s more, they did much extra work for using CS in low frequency part, too. In this paper, the existing achievements based on previ-ous contributions of watermark and CS [6-8] is improved. Firstly, the whole image is divided into many blocks, and the sub-blocks with larger visual capacity are selected to embed the watermark in CS domain considering the non-uniformity of the block based on HVS. Thus, the invisibility and the robustness of the watermark can be ensured. Then the coeffi-cient matrix is obtained by multiplying a sensing matrix to each sub-block. At last, the quantization steps are adjusted 2 The Open Cybernetics & Systemics Journal, 2015, Volume 9 Liao and Lv adaptably according to the non-uniformity property to mod-ify the coefficients bit by bit. Finally an image embedded with the watermark is generated. In this way, only a small amount of information is needed to reconstruct the original image perfectly. This greatly reduces the sampling time and data storage space. Moreover, CS has strong anti-interference ability that the remaining measurements still can reconstruct the original image, even if some measurements are lost. In addition, we embed the watermark in CS domain and make the measurement matrix as a key, it is hard to ex-tract or damage the watermark without the key due to the diversity of measurement matrix. Experimental results show that our proposed method improves the security and attack-resistance of the watermark greatly. The remainder of this paper is organized as follows. Compressive sensing theory is reviewed in Section 2. In Sec-tion 3, the proposed watermark embedding and extraction is introduced. Simulate results and analyses are described in Section 4. Section 5 concludes the work. 2. COMPRESSIVE SENSING THEORY 2.1. Compressive Sensing Traditional sample and encoding theory suffers from two inherent inefficiencies: The Shannon/Nyquist sampling theo-rem specifies that signal sampling rate should be at least two times of the signal bandwidth in order to avoid losing infor-mation. In many applications, including digital image and video, the Nyquist rate is so high that too many samples re-sult, which make compression necessary prior to storage and transmission. Nevertheless, in the process of compression, a large number of transform coefficients are discarded when the signal is compressible, thus causes greatly waste of the data computation and memory store resources. Different from conventional encoding and decoding the-ory, compressive sensing is new approach to represent com-pressive signals at a rate significantly below the Nyquist rate. CS employs non-adaptive linear projections that preserve the structure of the signal and the signal can be reconstructed from these projections using an optimization process. It does the process of signal sampling and compression coding si-multaneously. There are three processes in CS [6]--sparse representation of the signal, construction of measurement matrix and recon-struction of the signal. The core of CS is using a small amount of information to restore the original high-dimensional signals. Consider a real valued, finite length, one dimension, dis-crete signal x, which can be viewed as an Nx1 column vector in the RN. Any signal in RN can be represented as the linear combination of the column vectors in the basis matrix Ψ: x = Ψα (1) where α is the N x 1 column vector of weighting coeffi-cients. The signal x is K-sparse if only K of coefficients α are none-zero. The signal x is compressible if the representa-tion (1) has just a few large coefficients and many small co-efficients. Compressive sensing acquires a compressed representa-tion directly without experiencing the procedure of N sam-pling. Considering that general measurement process can be described as computing inner products between the signal x and a group of vectors which are arranged as the row vectors of the M×N (M<N) measurement matrix Φ. y = Φx (2) The measurement result is denoted as the M×1 column vector y. Then by substitution (1) in (2), y can be written as: y = Φ Ψα = Θα (3) where Θ is the sensing matrix. Now, the problem is that how to design a stable measurement matrix Φ such that the salient information in the signal x is not damaged during the dimen-sionality reduction from N to M. And another problem is a reconstruction algorithm to recover the signal x from the measurement y. This is a complex optimization problem because the process of getting x from y is an under-determined problem: min ||x||l1 s.t y = Θ x (4) Actually, this is a reconstructing question which is com-puting a minimum 1 norm of x with a constraint condition y = Θ x . 2.2. The Signal Reconstruction At present, the common reconstruction algorithms in-clude greedy pursuit algorithm and convex optimization al-gorithm [8, 9] et al. In this paper, the Orthogonal Matching Pursuit [8] (OMP) is adopted, which is a special greedy pur-suit algorithm with lower computation, better reconstruction ability, and easier realization. The process of OMP is de-scribed as follows: Input: Sensing matrix Θ, sampling vector α, sparse de-gree K; Output: The K-sparse approximation matrix )! of α, the reconstruction error r; Initialization: the residual r0 = y, index set J0 = Φ, itera-tions t=1; Step 1: Find the maximum value of the inner product of residual rt and the column of sensing matrix Θ: gt = Θtrt-1. Step 2: renew the index set Jt = Jt-1 ∪{ ρi}, the sensing matrix ΦJt = ΦJt-1 ∪{ ϕρi}. Step 3: solve !t = ("TJt"Jt)-1 "TJty by least-square method; Step 4: renew the residual rt = y- Θαt, t = t +1; Step 5: if t > K, stop the iteration, )! = αt, r = rt, else do step 1. 3. PROPOSED METHOD 3.1. Watermark Embedding The whole digital watermark algorithm includes embed-ding process and watermark extraction process, and the wa-termark embedding process includes the preprocessing of original watermark, the selection of embedding position, embedding formula and the reconstruction of the watermark. A Novel Watermark Embedding Scheme using Compressive Sensing The Open Cybernetics & Systemics Journal , 2015, Volume 9 3 The watermark extraction process can be viewed as the i n- verse process of watermark e mbedding. The basic model of this algorithm is given in Fig. (1). 3.1.1 . Watermark Image Scrambling For the sake of securit y and robustness of the watermark , Arnold algorithm is employed to scrambl e the 2 -D wate r- mark w(x, y) (16 ×16), the scrambling fo rmula is given as follows : x'y'!"#$%& = 11 1 2!"#$%& xy!"#$%& (mod 16) (5) where (x, y) and (x′, y′) represent the pixel coordinate s of original watermark w(x, y) and scrambled wate rmark w(x′, y′). w(x′, y′) is obtained by scrambled k times on w(x, y) . After scrambled, the watermark image is reshaped in to one- dimension vector by scann ing row by row: wn (i)(i = {0,1}, 1 ≤ i ≤ 16 × 16). 3.1.2 . Embedding Position Different image patch can be embedded in the watermark with different strength, and the strength of watermark will influence its robustness and invisibility. In the watermark system , these two features are always contradictory and only one feature could be guaranteed as large as possible on the basis of another feature inevitably in order to get a balance of both. For example, most of energy is concentrated in the low frequency. Any distortion in low frequency part is sensitive to human visual system (HVS). Therefore, it is not easy to keep its invisibility for us to e m- bed the watermar k in low frequency part. But the low fr e- quency part has strong sensory c apacity, which can be seen as strong background and accommodate more strenghful watermark information. In this way, the robustness of the watermark can be ensured . On the contrary, the HVS system is not sensitive to the distortion in high fr equency part, which can ensure the concealment of the watermark. But the embedding capacity of the high frequency part is low. So it’s difficult to guarantee the robustness of the watermark. Therefor e, whatever the watermark is embedded in the low frequency part or high frequency part, it is to enhance the robustness with the loss of concealment or vice versa . In this paper, a novel embedding strategy is proposed. Not taking the image as low frequenc y or high frequency globally, the whole image is divided into blocks and to search those blocks appropriate for embedding the wate r- mark based on some rules. As we know, the information distribution in the image is ununiformed. Some blocks co n- tain abundant texture inform ation, while other blocks are homogeneous. In comparison, the blocks with worse uni- formity have high visual capacity and are suitable for e m- bedding strength watermark. The measure of the non - uniformity in the block is given as follows: Assum ing the original image (M ×M) is dividing into the b×b blocks. The non -uniformity d(B) of every block is co m- puted by the formula ( 6): d(Bk) = 1b2 f(i,j) - mkm1+!ki,j "Bk# (6) Fig. (1). Proposed digital watermark algorithm . 4 The Open Cybernetics & Systemics Journal, 2015, Volume 9 Liao and Lv And k = 1,2 ...M!Mb!b, where Bk is the k-th sub-block, the mean of Bk is denoted as mk, τ is the weighting parame-ter. The larger value of d(Bk) means the more rich informa-tion, and the larger visual capacity. Then we choose the first L sub-blocks, which have larger capability to embed the wa-termark, so that the robustness of watermark can be ensured. Next, discrete wavelet transform (DWT) will be per-formed on each block to get a sparse representation. Then set the sampling rate as σ, and choose a Gaussian random matrix ΦL ((b*b*σ) ×(b*b)) as a sensing matrix to perform com-pressive sensing on the sparse wavelet coefficients, thus, we can get the measurements yi (i=1, 2,…,b*b*σ). Finally, the (b*b*σ)×L measurement matrix is obtained and take the sensing matrix ΦL as a key to be used for extracting the wa-termark. The detailed implementation of above procedure is de-scribed in Algorithm 1. Algorithm 1. (Compressive Sensing). Input: I(M×M); Output: yi; Initialization: iterations k=1, j=1,σ = 0.3, N=(m*m)/(b*b) I (M×M)→ {Bk} (k=1,2…N) For k=1 to N do calculate d(Bk); rank d(Bk) from larger to smaller ; choose former L(L<N) d(Bk) → Bk; End For j=1 to L do do DWT to Bk →xt (t = 1,2 …,b * b), Ψ ((b * b) × (b * b)) is the transform matrix; randn (b * b * σ) × (b * b)→ sensing matrix Φj, save Φj; yi = Φ × Ψ × xt; j=j+1! End 3.1.3. Embedding Algorithm After the measurement matrix is obtained, the watermark information is embedded into the measurement matrix through quantization method. Due to the size of the water-mark is b*b, it is necessary to repeat embedding the same watermark K times ( K = L *b*b*sb*b = L*s). According to the quantization principle, the measurements yi is modified bit by bit. The embedding formula is as follows: λi = round(yi/δ) (7) δ = a * d(Bk) (8) )yi = (!i " 1/2) # $, mod(!i + wn, 2) = 1(!i + 1/2) # $, mod(!i + wn, 2) = 0%&'(' (9) where yi is the measurement before embedded, and )yi is the measurement after embedded. The “round” function is used to get the nearest integer, and the “mod” function represents module operation. In addition, δ is the quantization step, which depends on the value of d(Bk), and a is constant. The larger of d(Bk), the larger of date volume and quantization step, then we can embed watermark with more capacity, and therefore the quantization step is adaptable here. Through previous theory proof [9], the equation (9) has optimality. In this way, a watermark with high security, robustness and invisibility can be achieved. Finally, after embedding watermark into the carrier im-age, each block is reconstructed by the OMP algorithm men-tioned in section 2. Then Inverse Discrete Wavelet Trans-form (IDWT) will be performed for whole image. Thus, the image embedded with watermark )I is obtained. 3.2. Watermark Extraction The process of watermark extraction isn’t ready to dis-cuss in details, considering that the extraction is the inverse process of the watermark embedding. So simple summariza-tion is given as follows: segment the image into the 16x16 blocks, then compute the non-uniformity of each block based on formula (6) and choose the first L blocks of larger non-uniformity to perform compressive sensing according to the key ΦL. Then the watermark is extracted as follows: )wn = 1 ! mod()!i,2) = 10 ! mod()!i,2) = 0"#$%$ (10) where )!i = floor ()yi/"), floor is the downward integral function. 4. SIMULATION RESULTS AND ANALYSIS Although CS has been applied successfully in watermark recently, but most of them are not blind watermark. In this paper, a novel blind watermark embedding algorithm based on CS (BWCS) is proposed. In order to verify the perform-ance of proposed method, the following experimental results will show that a clear embedding image can be obtained even under a low sampling rate, and the proposed method can resist some common attacks in most cases. Given a 16x16 watermark image, its numerical bits are embedded into the gray carrier image Lena (256x256) or Baboon (256x256), using the watermark embedding algo-rithm introduced in Section 3. The sampling rate σ is set as 0.3 and the block size b is set as 16. The experiment results are shown in Fig. (2). We can hardly distinguish the differ-ence between the original and embedding image visually. This indicates the high invisibility of the watermark. 4.1. Robustness Analysis To simulate the communication conditions and deliberate or unintentional processing, some attacks need to be used in the watermarked image to test the robustness of a watermark system. Experiments are performed under the filter attack, noise attack or compression attack. The results are compared with traditional watermarking methods based on DWT trans-form domain (DWT) [10] and non-adaptive quantization watermarking method (NAQW) [11]. A Novel Watermark Embedding Scheme using Compressive Sensing The Open Cybernetics & Systemics Journal , 2015, Volume 9 5 Fig. (2). Watermark embedding results. (1) Filtering Attack . The watermarked image is filtered by G aussian Low-Pass Filter (GLPF) , shown in Fig. ( 3a). The extracted watermarks by three methods are given in Figs. ( 3b-d). Although the extracted watermark s are not so clear, the key i nformation in Fig. ( 3d) is still clear. The result is better than other two methods . (2) Noise Attack . The salt and pepper noise with density of 0.02 is added into the watermark ed im age, the extracted watermark s are showed in Figs. ( 4c-d). Due to the noise i n- terference , some information maybe lost when extracting the watermark through sparse decomposition of the watermarked image . But a clear watermark is still obtain ed, which is bett er than that of other two methods . Therefore, the watermark embedding algorithm proposed in this paper has better anti - noise ability. (3) Compression Attack . The watermarked image is compressed by JPEG algorithm and the recovered wate r- marked image is shown in Fig. ( 5a). It is obvious in Fig. ( 5d) that the extracting watermark by our proposed method is relative clear even though some information is lost during quan tization coding , the quality of which is superior to DWT and NAQW methods . It is enough to demo nstrate that the image embedding with watermark by the algorithm in this paper can resist against the JPEG compressing in a great degree . 4.2. Invisibility Analysis The in visibil ity of watermark is evaluated by the stru c- ture similarity index ( SSIM ) criterion [12] . SSIM reflects the simila rity of original image and the embedding image. The larger of the SSIM value, the higher of the watermark quality and the better of the watermark invisibility. The formula is showed as follows: SIMM(I,)I) = L(I,)I) ! C(I,)I) ! S(I,)I) (11) where the brightness similarity function, L (I,)I) = 2µ(I) * µ()I) + c1µ2(I) + µ2()I) + c1 (12) µ(!) is the average of an image. And the contrast simila r- ity function, C (I,)I) = 2!(I) * !()I) + c2!2(I) + !2()I) + c2 (13) !(") is the variance of an image. And the structure simi- larity function , S (I,)I) = cov(I,)I) + c3cov(I) ! cov()I) + c3 (14) cov (I,)I) is the covarian ce between the original image I and the watermarked image )I. The Tab le 1 shows the invis i-Fig. (3). Filtering Attack. Fig. (4). Noise Attack. Fig. (5). Compression attack. ( ( a ) L e n a (c) watermark e (e) original water m ed Lena mark (f) the extr a watermark fro m (b) Baboon ( d ) w a t e r m a r k ac t e d ( g ) t m Lena water m ked Baboon the extracted mark from Baboon 6 The Open Cybernetics & Systemics Journal, 2015, Volume 9 Liao and Lv bility of watermark by the proposed method BWCS in com-parison with DWT and NAQW. It can be easily seen that the SSIM of the BWCS is higher than that of DWT or NAQW, which means that there is higher invisibility of the watermark in this paper than other two methods. Table 1. Comparison of SSIM. Original Image BWC S DWT NAQW Lena 0.94 0.88 0.91 Baboon 0.93 0.86 0.91 4.3. Computation Complexity Analysis The time cost of an algorithm is an important measure of the computation complexity. The Table 2 gives the compari-son of running time embedding the same watermark into the “Lena” and “Baboon” image, using three different methods respectively. Table 2. Running time comparison. Original Image BWC S DWT NAQW Lena 11.31s 18.74s 11.25s Baboon 11.02s 18.81s 10.97s We can find the running time of the algorithm in this pa-per is obviously less than the DWT, but comparable with the NAQW. CONCLUSION In this paper, an improved digital watermarking algo-rithm, which performs digital watermark embedding process in CS domain, is proposed. Experimental results show that the algorithm obtains robust and invisible embedded water-mark with larger capacity of data. At the same time, the abil-ity of defending against attack and extraction of embedded watermark is greatly improved. The watermark is detected without any reference to the original image. This remarkably decreases the costs of storing carrier data and the computa-tion complexity. We can achieve a watermark with high se-curity, strong ability of defending against attack and extrac-tion due to the irreversibility of CS. CONFLICT OF INTEREST The authors confirm that this article content has no con-flict of interest. ACKNOWLEDGEMENTS The authors would like to appreciate the anonymous re-viewers for their valuable comments and suggestions. This work is supported by the Fundamental Research Funds for the Central Universities (No. NCEPU2014MS02). REFERENCES [1] V.R. Schydel, A. Triklel, and C. Osbome, “A digital watermark,” In: Proceedings of International Conference on Image Processing. Texas: IEEE, 1994, vol. 2, pp. 86-88. [2] E.J. Candes, and M.B. Wakin, “An introduction to compressive sampling,” IEEE Signal Proc. Mag., vol. 25, no.2, pp. 21-30, 2008. [3] H.C. Huang, F.C. Chang, and C.H. Wu, “Watermarking for com-pressive sampling applications,” In: Proceedings of the 8th Interna-tional Conference on Intelligent Information on Hiding and Multi-media Signal Processing, Washington: IEEE, 2012, pp. 223-226. [4] W.J. Lin, Reconstruction algorithm for compressive sensing and their applications to digital watermarking, Jiaotong University: Beijing, 2011. [5] F. Wei, D. Liang, C. Zhang, and W.X. Bao, “Watermarking algo-rithm for digital image based on compressive sensing measure-ments,” J. Anhui Univ., vol. 1000-2162, no. 3, pp. 61-68, 2013. [6] R. Baraniuk, “Compressive sensing,” IEEE Signal. Proc. Mag., vol. 24, no.4, pp. 118-121, 2007. [7] G.M. Shi, D.H. Liu, D.H. Gao, Z. Liu, J. Lin, and L.J. Wang, “Ad-vances in theory of compressed sensing,” Acta Electron. Sin., vol. 37, no.5, pp. 1070-1081, 2009. [8] J. Shihao, X. Ya, and L. Carin. “Bayesian compressive sensing,” IEEE Signal Proc. Mag., vol. 56, no.6, pp. 2346-2356, 2008. [9] Li X D, “Optimization analysis of formulas quantization-based image watermarking,” Opto-Electron. Eng., vol. 37, no. 2, pp. 96-102, 2010. [10] Y.J. Cai, Y. Niu, and S.U. Qing, “Blind watermarking algorithm for images based on DWT-SVD and Fibonacci transformation,” Appli-cation Res. Comput., vol. 29, no.8, pp. 3025-3028, 2012. [11] H. Xu, and C.Z. Xiong, “Analysis of quantization-based water-marking,” J. Communi., vol. 27, no. 3, pp. 15-27, 2006. [12] Z. Wang, A.C. Bovik, H.R. Sheikh and E.P. Simoncelli, “Image quality assessment: from error visibility to structure similarity,” IEEE Trans. Image Process, vol. 13, no.4, pp. 600-612, 2004. Received: September 12, 2014 Revised: October 19, 2014 Accepted: October 24, 2014 © Liao and Lv; Licensee Bentham Open. This is an open access article licensed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/-licenses/by-nc/3.0/) which permits unrestricted, non-commercial use, distribution and reproduction in any medium, provided the work is properly cited.

You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you s... You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you select really corresponds to the excerpt: there will be details mentioned in the excerpt that should appear in the paper. If you read an abstract and it seems like it could be the paper we’re looking for, read the paper to make sure. Also: sometimes you’ll read a paper that cites the paper we’re looking for. In such cases, please go to the references in order to find the full name of the paper we’re looking for, and search for it, and then select it. [CITATION] empirically study calibration under distribution shift and provide a large comparison of methods for improving calibration. They report that both accuracy and calibration deteriorate with distribution shift. You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you select really corresponds to the excerpt: there will be details mentioned in the excerpt that should appear in the paper. If you read an abstract and it seems like it could be the paper we’re looking for, read the paper to make sure. Also: sometimes you’ll read a paper that cites the paper we’re looking for. In such cases, please go to the references in order to find the full name of the paper we’re looking for, and search for it, and then select it. [CITATION] empirically study calibration under distribution shift and provide a large comparison of methods for improving calibration. They report that both accuracy and calibration deteriorate with distribution shift.

- Paper ID: 48e09f69cfc379f66f7030b0c142fb9eb8284e1e Title: Towards Reliable Multimodal Stress Detection under Distribution Shift Abst... - Paper ID: 48e09f69cfc379f66f7030b0c142fb9eb8284e1e Title: Towards Reliable Multimodal Stress Detection under Distribution Shift Abstract: The recognition of stress is an important issue from a health care perspective as well as in the human-computer interaction context. With the help of multimodal sensors, stress can be detected relatively well under laboratory conditions. However, when models are used in the real world, shifts in the data distribution can occur, often leading to performance degradation. It is therefore desirable that models in these scenarios are at least able to accurately capture this uncertainty and thus know what they do not know. This work aims to investigate how synthetic shifts in the data distribution can affect the reliability of a multimodal stress detection model in terms of calibration and uncertainty quantification. We compare a baseline with three known approaches that aim to improve reliability of uncertainty estimates. Our results show that all... - Paper ID: 48e09f69cfc379f66f7030b0c142fb9eb8284e1e Title: Towards Reliable Multimodal Stress Detection under Distribution Shift Abstract: The recognition of stress is an important issue from a health care perspective as well as in the human-computer interaction context. With the help of multimodal sensors, stress can be detected relatively well under laboratory conditions. However, when models are used in the real world, shifts in the data distribution can occur, often leading to performance degradation. It is therefore desirable that models in these scenarios are at least able to accurately capture this uncertainty and thus know what they do not know. This work aims to investigate how synthetic shifts in the data distribution can affect the reliability of a multimodal stress detection model in terms of calibration and uncertainty quantification. We compare a baseline with three known approaches that aim to improve reliability of uncertainty estimates. Our results show that all methods we tested improve the calibration. However, calibration generally deteriorates and spreads with stronger shifts for all approaches. They perform especially poorly for shifts in highly relevant modalities. Overall, we conclude that in the conducted experiments the investigated methods are not sufficiently reliable under distribution shifts. Citation Count: 5 - Paper ID: 771374d3368116fc0f8acba0955769401381d831 Title: Exploring Covariate and Concept Shift for Detection and Calibration of Out-of-Distribution Data Abstract: Moving beyond testing on in-distribution data works on Out-of-Distribution (OOD) detection have recently increased in popularity. A recent attempt to categorize OOD data introduces the concept of near and far OOD detection. Specifically, prior works define characteristics of OOD data in terms of detection difficulty. We propose to characterize the spectrum of OOD data using two types of distribution shifts: covariate shift and concept shift, where covariate shift corresponds to change in style, e.g., noise, and concept shift indicates a change in semantics. This characterization reveals that sensitivity to each type of shift is important to the detection and confidence calibration of OOD data. Consequently, we investigate score functions that capture sensitivity to each type of dataset shift and methods that improve them. To this end, we theoretically derive two score functions for OOD detection, the covariate shift score and concept shift score, based on the decomposition of KL-divergence for both scores, and propose a geometrically-inspired method (Geometric ODIN) to improve OOD detection under both shifts with only in-distribution data. Additionally, the proposed method naturally leads to an expressive post-hoc calibration function which yields state-of-the-art calibration performance on both in-distribution and out-of-distribution data. We are the first to propose a method that works well across both OOD detection and calibration and under different types of shifts. View project page at https://sites.google.com/view/geometric-decomposition. Citation Count: 3 - Paper ID: 8373e1623e578b3612494234b34a3b9f9eb177b8 Title: Frustratingly Easy Uncertainty Estimation for Distribution Shift Abstract: Distribution shift is an important concern in deep image classification, produced either by corruption of the source images, or a complete change, with the solution involving domain adaptation. While the primary goal is to improve accuracy under distribution shift, an important secondary goal is uncertainty estimation: evaluating the probability that the prediction of a model is correct. While improving accuracy is hard, uncertainty estimation turns out to be frustratingly easy. Prior works have appended uncertainty estimation into the model and training paradigm in various ways. Instead, we show that we can estimate uncertainty by simply exposing the original model to corrupted images, and performing simple statistical calibration on the image outputs. Our frustratingly easy methods demonstrate superior performance on a wide range of distribution shifts as well as on unsupervised domain adaptation tasks, measured through extensive experimentation. Citation Count: 1 - Paper ID: d78bd5a195546b56a9b0f91e91a1f818de9aab1c Title: Confidence Calibration for Domain Generalization under Covariate Shift Abstract: Existing calibration algorithms address the problem of covariate shift via unsupervised domain adaptation. However, these methods suffer from the following limitations: 1) they require unlabeled data from the target domain, which may not be available at the stage of calibration in real-world applications and 2) their performance depends heavily on the disparity between the distributions of the source and target domains. To address these two limitations, we present novel calibration solutions via domain generalization. Our core idea is to leverage multiple calibration domains to reduce the effective distribution disparity between the target and calibration domains for improved calibration transfer without needing any data from the target domain. We provide theoretical justification and empirical experimental results to demonstrate the effectiveness of our proposed algorithms. Compared against state-of-the-art calibration methods designed for domain adaptation, we observe a decrease of 8.86 percentage points in expected calibration error or, equivalently, an increase of 35 percentage points in improvement ratio for multi-class classification on the Office-Home dataset. Citation Count: 22 - Paper ID: 51bee70b4a53baa635f8b4fdd523e62410ce924c Title: System Performance as a Function of Calibration Methods, Sample Size and Sampling Variability in Likelihood Ratio-Based Forensic Voice Comparison Abstract: In data-driven forensic voice comparison, sample size is an issue which can have substantial effects on system output. Numerous calibration methods have been developed and some have been proposed as solutions to sample size issues. In this paper, we test four calibration methods (i.e. logistic regression, regularised logistic regression, Bayesian model, ELUB) under different conditions of sampling variability and sample size. Training and test scores were simulated from skewed distributions derived from real experiments, increasing sample sizes from 20 to 100 speakers for both the training and test sets. For each sample size, the experiments were replicated 100 times to test the susceptibility of different calibration methods to sampling variability. The C llr mean and range across replications were used for evaluation. The Bayesian model and regularized logistic regression produced the most stable C llr values when the sample size is small (i.e. 20 speakers), although mean C llr is consistently lowest using logistic regression. The ELUB calibration method generally is the least preferred as it is the most sensitive to sample size and sampling variability (mean = 0.66, range = 0.21-0.59). Citation Count: 4 - Paper ID: 115e5a3946951b4a8cbb776cb55bedf6a22c0773 Title: Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift Abstract: Modern neural networks have proven to be powerful function approximators, providing state-of-the-art performance in a multitude of applications. They however fall short in their ability to quantify confidence in their predictions - this is crucial in high-stakes applications that involve critical decision-making. Bayesian neural networks (BNNs) aim at solving this problem by placing a prior distribution over the network's parameters, thereby inducing a posterior distribution that encapsulates predictive uncertainty. While existing variants of BNNs based on Monte Carlo dropout produce reliable (albeit approximate) uncertainty estimates over in-distribution data, they tend to exhibit over-confidence in predictions made on target data whose feature distribution differs from the training data, i.e., the covariate shift setup. In this paper, we develop an approximate Bayesian inference scheme based on posterior regularisation, wherein unlabelled target data are used as "pseudo-labels" of model confidence that are used to regularise the model's loss on labelled source data. We show that this approach significantly improves the accuracy of uncertainty quantification on covariate-shifted data sets, with minimal modification to the underlying model architecture. We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations. Citation Count: 29 - Paper ID: 5e5848aa7f0d37e7ead301fbb17c7531107db4e4 Title: Maximum Likelihood with Bias-Corrected Calibration is Hard-To-Beat at Label Shift Adaptation Abstract: Label shift refers to the phenomenon where the prior class probability p ( y ) changes between the training and test distributions, while the conditional probability p ( x | y ) stays fixed. Label shift arises in settings like medical diagnosis, where a classifier trained to predict disease given symptoms must be adapted to scenarios where the base-line prevalence of the disease is different. Given estimates of p ( y | x ) from a predictive model, Saerens et al. proposed an efficient maximum likelihood algorithm to correct for label shift that does not require model retraining, but a limiting assumption of this algorithm is that p ( y | x ) is calibrated, which is not true of modern neural networks. Recently, Black Box Shift Learning (BBSL) and Regularized Learning under Label Shifts (RLLS) have emerged as state-of-the-art techniques to cope with label shift when a classi-fier does not output calibrated probabilities, but both methods require model retraining with importance weights and neither has been benchmarked against maximum likelihood. Here we (1) show that combining maximum likelihood with a type of calibration we call bias-corrected calibration outperforms both BBSL and RLLS across diverse datasets and distribution shifts, (2) prove that the maximum likelihood objective is concave, and (3) introduce a principled strategy for estimating source-domain priors that improves robustness to poor calibration. This work demonstrates that the maximum likelihood with appropriate calibration is a formidable and efficient baseline for label shift adaptation; notebooks reproducing experiments available https://github.com/ Citation Count: 73 - Paper ID: f891d0e41857e8b214616e199acdc906349091d9 Title: A Novel Unsupervised Post-Processing Calibration Method for DNNS with Robustness to Domain Shift Abstract: The uncertainty estimation is critical in real-world decision making applications, especially when distributional shift between the training and test data are prevalent. Many calibration methods in the literature have been proposed to improve the predictive uncertainty of DNNs which are generally not well-calibrated. However, none of them is specifically designed to work properly under domain shift condition. In this paper, we propose Unsupervised Temperature Scaling (UTS) as a robust calibration method to domain shift. It exploits unlabeled test samples instead of the training one to adjust the uncertainty prediction of deep models towards the test distribution. UTS utilizes a novel loss function, weighted NLL, which allows unsupervised calibration. We evaluate UTS on a wide range of model-datasets to show the possibility of calibration without labels and demonstrate the robustness of UTS compared to other methods (e.g., TS, MC-dropout, SVI, ensembles) in shifted domains. Citation Count: 0 - Paper ID: 1eb7f46b1a0a7df823194d86543e5554aa21021a Title: Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift Abstract: Modern machine learning methods including deep learning have achieved great success in predictive accuracy for supervised learning tasks, but may still fall short in giving useful estimates of their predictive {\em uncertainty}. Quantifying uncertainty is especially critical in real-world settings, which often involve input distributions that are shifted from the training distribution due to a variety of factors including sample bias and non-stationarity. In such settings, well calibrated uncertainty estimates convey information about when a model's output should (or should not) be trusted. Many probabilistic deep learning methods, including Bayesian-and non-Bayesian methods, have been proposed in the literature for quantifying predictive uncertainty, but to our knowledge there has not previously been a rigorous large-scale empirical comparison of these methods under dataset shift. We present a large-scale benchmark of existing state-of-the-art methods on classification problems and investigate the effect of dataset shift on accuracy and calibration. We find that traditional post-hoc calibration does indeed fall short, as do several other previous methods. However, some methods that marginalize over models give surprisingly strong results across a broad spectrum of tasks. Citation Count: 1389 - Paper ID: 6c48095f7883d726dd2679394c79c1c955b2aaa1 Title: Source-Free Adaptation to Measurement Shift via Bottom-Up Feature Restoration Abstract: Source-free domain adaptation (SFDA) aims to adapt a model trained on labelled data in a source domain to unlabelled data in a target domain without access to the source-domain data during adaptation. Existing methods for SFDA leverage entropy-minimization techniques which: (i) apply only to classification; (ii) destroy model calibration; and (iii) rely on the source model achieving a good level of feature-space class-separation in the target domain. We address these issues for a particularly pervasive type of domain shift called measurement shift which can be resolved by restoring the source features rather than extracting new ones. In particular, we propose Feature Restoration (FR) wherein we: (i) store a lightweight and flexible approximation of the feature distribution under the source data; and (ii) adapt the feature-extractor such that the approximate feature distribution under the target data realigns with that saved on the source. We additionally propose a bottom-up training scheme which boosts performance, which we call Bottom-Up Feature Restoration (BUFR). On real and synthetic data, we demonstrate that BUFR outperforms existing SFDA methods in terms of accuracy, calibration, and data efficiency, while being less reliant on the performance of the source model in the target domain. Citation Count: 45

Can You Trust Your Model’s Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift Yaniv Ovadia Google Research yovadia@google.c... Can You Trust Your Model’s Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift Yaniv Ovadia Google Research yovadia@google.comEmily Fertigy Google Research emilyaf@google.comJie Reny Google Research jjren@google.com Zachary Nado Google Research znado@google.comD Sculley Google Research dsculley@google.comSebastian Nowozin Google Research nowozin@google.com Joshua V . Dillon Google Research jvdillon@google.comBalaji Lakshminarayananz DeepMind balajiln@google.comJasper Snoekz Google Research jsnoek@google.com Abstract Modern machine learning methods including deep learning have achieved great success in predictive accuracy for supervised learning tasks, but may still fall short in giving useful estimates of their predictive uncertainty . Quantifying uncertainty is especially critical in real-world settings, which often involve input distributions that are shifted from the training distribution due to a variety of factors including sample bias and non-stationarity. In suc... Can You Trust Your Model’s Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift Yaniv Ovadia Google Research yovadia@google.comEmily Fertigy Google Research emilyaf@google.comJie Reny Google Research jjren@google.com Zachary Nado Google Research znado@google.comD Sculley Google Research dsculley@google.comSebastian Nowozin Google Research nowozin@google.com Joshua V . Dillon Google Research jvdillon@google.comBalaji Lakshminarayananz DeepMind balajiln@google.comJasper Snoekz Google Research jsnoek@google.com Abstract Modern machine learning methods including deep learning have achieved great success in predictive accuracy for supervised learning tasks, but may still fall short in giving useful estimates of their predictive uncertainty . Quantifying uncertainty is especially critical in real-world settings, which often involve input distributions that are shifted from the training distribution due to a variety of factors including sample bias and non-stationarity. In such settings, well calibrated uncertainty estimates convey information about when a model’s output should (or should not) be trusted. Many probabilistic deep learning methods, including Bayesian-and non- Bayesian methods, have been proposed in the literature for quantifying predictive uncertainty, but to our knowledge there has not previously been a rigorous large- scale empirical comparison of these methods under dataset shift. We present a large- scale benchmark of existing state-of-the-art methods on classification problems and investigate the effect of dataset shift on accuracy and calibration. We find that traditional post-hoc calibration does indeed fall short, as do several other previous methods. However, some methods that marginalize over models give surprisingly strong results across a broad spectrum of tasks. 1 Introduction Recent successes across a variety of domains have led to the widespread deployment of deep neural networks (DNNs) in practice. Consequently, the predictive distributions of these models are increasingly being used to make decisions in important applications ranging from machine-learning aided medical diagnoses from imaging (Esteva et al., 2017) to self-driving cars (Bojarski et al., 2016). Such high-stakes applications require not only point predictions but also accurate quantification of predictive uncertainty, i.e. meaningful confidence values in addition to class predictions. With sufficient independent labeled samples from a target data distribution, one can estimate how well Equal contribution yAI Resident zCorresponding authors 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.arXiv:1906.02530v2 [stat.ML] 17 Dec 2019a model’s confidence aligns with its accuracy and adjust the predictions accordingly. However, in practice, once a model is deployed the distribution over observed data may shift and eventually be very different from the original training data distribution. Consider, e.g., online services for which the data distribution may change with the time of day, seasonality or popular trends. Indeed, robustness under conditions of distributional shift and out-of-distribution (OOD) inputs is necessary for the safe deployment of machine learning (Amodei et al., 2016). For such settings, calibrated predictive uncertainty is important because it enables accurate assessment of risk, allows practitioners to know how accuracy may degrade, and allows a system to abstain from decisions due to low confidence. A variety of methods have been developed for quantifying predictive uncertainty in DNNs. Probabilis- tic neural networks such as mixture density networks (MacKay & Gibbs, 1999) capture the inherent ambiguity in outputs for a given input, also referred to as aleatoric uncertainty (Kendall & Gal, 2017). Bayesian neural networks learn a posterior distribution over parameters that quantifies parameter uncertainty, a type of epistemic uncertainty that can be reduced through the collection of additional data. Popular approximate Bayesian approaches include Laplace approximation (MacKay, 1992), variational inference (Graves, 2011; Blundell et al., 2015), dropout-based variational inference (Gal & Ghahramani, 2016; Kingma et al., 2015), expectation propagation Hern ´andez-Lobato & Adams (2015) and stochastic gradient MCMC (Welling & Teh, 2011). Non-Bayesian methods include training multiple probabilistic neural networks with bootstrap or ensembling (Osband et al., 2016; Lakshminarayanan et al., 2017). Another popular non-Bayesian approach involves re-calibration of probabilities on a held-out validation set through temperature scaling (Platt, 1999), which was shown by Guo et al. (2017) to lead to well-calibrated predictions on the i.i.d. test set. Using Distributional Shift to Evaluate Predictive Uncertainty While previous work has evaluated the quality of predictive uncertainty on OOD inputs (Lakshminarayanan et al., 2017), there has not to our knowledge been a comprehensive evaluation of uncertainty estimates from different methods under dataset shift. Indeed, we suggest that effective evaluation of predictive uncertainty is most meaningful under conditions of distributional shift. One reason for this is that post-hoc calibration gives good results in independent and identically distributed (i.i.d.) regimes, but can fail under even a mild shift in the input data. And in real world applications, as described above, distributional shift is widely prevalent. Understanding questions of risk, uncertainty, and trust in a model’s output becomes increasingly critical as shift from the original training data grows larger. Contributions In the spirit of calls for more rigorous understanding of existing methods (Lipton & Steinhardt, 2018; Sculley et al., 2018; Rahimi & Recht, 2017), this paper provides a benchmark for evaluating uncertainty that focuses not only on the i.i.d. setting but also uncertainty under distributional shift. We present a large-scale evaluation of popular approaches in probabilistic deep learning, focusing on methods that operate well in large-scale settings, and evaluate them on a diverse range of classification benchmarks across image, text, and categorical modalities. We use these experiments to evaluate the following questions: How trustworthy are the uncertainty estimates of different methods under dataset shift? Does calibration in the i.i.d. setting translate to calibration under dataset shift? How do uncertainty and accuracy of different methods co-vary under dataset shift? Are there methods that consistently do well in this regime? In addition to answering the questions above, our code is made available open-source along with our model predictions such that researchers can easily evaluate their approaches on these benchmarks4. 2 Background Notation and Problem Setup Letx2Rdrepresent a set of d-dimensional features and y2 f1;:::;kgdenote corresponding labels (targets) for k-class classification. We assume that a training datasetDconsists ofNi.i.d.samplesD=f(xn;yn)gN n=1. Letp(x;y)denote the true distribution (unknown, observed only through the samples D), also referred to as the data generating process . We focus on classification problems, in which the true distribution is assumed to be a discrete distribution over kclasses, and the observed y2f1;:::;kg 4https://github.com/google-research/google-research/tree/master/uq benchmark 2019 2is a sample from the conditional distribution p(yjx). We use a neural network to model p(yjx)and estimate the parameters using the training dataset. At test time, we evaluate the model predictions against a test set, sampled from the same distribution as the training dataset. However, here we also evaluate the model against OOD inputs sampled from q(x;y)6=p(x;y). In particular, we consider two kinds of shifts: shifted versions of the test inputs where the ground truth label belongs to one of the kclasses. We use shifts such as corruptions and perturbations proposed by Hendrycks & Dietterich (2019), and ideally would like the model predictions to become more uncertain with increased shift, assuming shift degrades accuracy. This is also referred to as covariate shift (Sugiyama et al., 2009). a completely different OOD dataset , where the ground truth label is not one of the kclasses. Here we check if the model exhibits higher predictive uncertainty for those new instances and to this end report diagnostics that rely only on predictions and not ground truth labels. High-level overview of existing methods A large variety of methods have been developed to either provide higher quality uncertainty estimates or perform OOD detection to inform model confidence. These can roughly be divided into: 1. Methods which deal with p(yjx)only, we discuss these in more detail in Section 3. 2.Methods which model the joint distribution p(y;x), e.g. deep hybrid models (Kingma et al., 2014; Alemi et al., 2018; Nalisnick et al., 2019; Behrmann et al., 2018). 3.Methods with an OOD-detection component in addition to p(yjx)(Bishop, 1994; Lee et al., 2018; Liang et al., 2018), and related work on selective classification (Geifman & El-Yaniv, 2017). We refer to Shafaei et al. (2018) for a recent summary of these methods. Due to the differences in modeling assumptions, a fair comparison between these different classes of methods is challenging; for instance, some OOD detection methods rely on knowledge of a known OOD set, or train using a none-of-the-above class, and it may not always be meaningful to compare predictions from these methods with those obtained from a Bayesian DNN. We focus on methods described by (1) above, as this allows us to focus on methods which make the same modeling assumptions about data and differ only in how they quantify predictive uncertainty. 3 Methods and Metrics We select a subset of methods from the probabilistic deep learning literature for their prevalence, scalability and practical applicability5. These include (see also references within): (Vanilla ) Maximum softmax probability (Hendrycks & Gimpel, 2017) (Temp Scaling ) Post-hoc calibration by temperature scaling using a validation set (Guo et al., 2017) (Dropout ) Monte-Carlo Dropout (Gal & Ghahramani, 2016; Srivastava et al., 2015) with rate p (Ensembles ) Ensembles of Mnetworks trained independently on the entire dataset using random initialization (Lakshminarayanan et al., 2017) (we set M= 10 in experiments below) (SVI) Stochastic Variational Bayesian Inference for deep learning (Blundell et al., 2015; Graves, 2011; Louizos & Welling, 2017, 2016; Wen et al., 2018). We refer to Appendix A.6 for details of our SVI implementation. (LL) Approx. Bayesian inference for the parameters of the last layer only (Riquelme et al., 2018) –(LL SVI ) Mean field stochastic variational inference on the last layer only –(LL Dropout ) Dropout only on the activations before the last layer In addition to metrics (we use arrows to indicate which direction is better) that do not depend on predictive uncertainty, such as classification accuracy ", the following metrics are commonly used: 5The methods used scale well for training and prediction (see in Appendix A.9.) . We also explored methods such as scalable extensions of Gaussian Processes (Hensman et al., 2015), but they were challenging to train on the 37M example Criteo dataset or the 1000 classes of ImageNet. 3Negative Log-Likelihood (NLL) #Commonly used to evaluate the quality of model uncertainty on some held out set. Drawbacks: Although a proper scoring rule (Gneiting & Raftery, 2007), it can over-emphasize tail probabilities (Quinonero-Candela et al., 2006). Brier Score#(Brier, 1950) Proper scoring rule for measuring the accuracy of predicted probabilities. It is computed as the squared error of a predicted probability vector ,p(yjxn;), and the one-hot encoded true response, yn. That is, BS =jYj1X y2Y(p(yjxn;)(yyn))2=jYj1 12p(ynjxn;) +X y2Yp(yjxn;)2 :(1) The Brier score has a convenient interpretation as BS= uncertaintyresolution + reliability , where uncertainty is the marginal uncertainty over labels, resolution measures the deviation of individual predictions against the marginal, and reliability measures calibration as the average violation of long-term true label frequencies. We refer to DeGroot & Fienberg (1983) for the decomposition of Brier score into calibration and refinement for classification and to (Br ¨ocker, 2009) for the general decomposition for any proper scoring rule. Drawbacks: Brier score is insensitive to predicted probabilities associated with in/frequent events. Both the Brier score and the negative log-likelihood are proper scoring rules and therefore the optimum score corresponds to a perfect prediction. In addition to these two metrics, we also evaluate two metrics— expected calibration error andentropy . Neither of these is a proper scoring rule, and thus there exist trivial solutions which yield optimal scores; for example, returning the marginal probabilityp(y)for every instance will yield perfectly calibrated but uninformative predictions. Each proper scoring rule induces a calibration measure (Br ¨ocker, 2009). However, ECE is not the result of such decomposition and has no corresponding proper scoring rule; we instead include ECE because it is popularly used and intuitive. Each proper scoring rule is also associated with a corresponding entropy function and Shannon entropy is that for log probability (Gneiting & Raftery, 2007). Expected Calibration Error (ECE) #Measures the correspondence between predicted probabilities and empirical accuracy (Naeini et al., 2015). It is computed as the average gap between within bucket accuracy and within bucket predicted probability for SbucketsBs=fn21:::N : p(ynjxn;)2(s;s+1]g. That is, ECE =PS s=1jBsj Njacc(Bs)conf(Bs)j;where acc(Bs) = jBsj1P n2Bs[yn= ^yn],conf(Bs) =jBsj1P n2Bsp(^ynjxn;), and ^yn= arg max yp(yjxn;) is then-th prediction. When bins fs:s21:::Sgare quantiles of the held-out predicted probabilities,jBsjjBkjand the estimation error is approximately constant. Drawbacks: Due to binning, ECE does not monotonically increase as predictions approach ground truth. If jBsj6=jBkj, the estimation error varies across bins. There is no ground truth label for fully OOD inputs. Thus we report histograms of confidence and predictive entropy on known and OOD inputs and accuracy versus confidence plots (Laksh- minarayanan et al., 2017): Given the prediction p(y=kjxn;), we define the predicted label as ^yn= arg max yp(yjxn;), and the confidence as p(y= ^yjx;) = max kp(y=kjxn;). We filter out test examples corresponding to a particular confidence threshold 2[0;1]and compute the accuracy on this set. 4 Experiments and Results We evaluate the behavior of the predictive uncertainty of deep learning models on a variety of datasets across three different modalities: images, text and categorical (online ad) data. For each we follow standard training, validation and testing protocols, but we additionally evaluate results on increasingly shifted data and an OOD dataset. We detail the models and implementations used in Appendix A. Hyperparameters were tuned for all methods using Bayesian optimization (Golovin et al., 2017) (except on ImageNet) as detailed in Appendix A.8. 4.1 An illustrative example - MNIST We first illustrate the problem setup and experiments using the MNIST dataset. We used the LeNet (LeCun et al., 1998) architecture, and, as with all our experiments, we follow standard training, validation, testing and hyperparameter tuning protocols. However, we also compute predictions on increasingly shifted data (in this case increasingly rotated or horizontally translated images) and study 40.10.20.30.40.50.60.70.80.91.0Accuracy ValidTest15± 30± 45± 60± 75± 90± 105± 120± 135± 150± 165± 180± Intensity of Shift0.00.20.40.60.81.01.21.41.6Brier(a) Rotated MNIST 0.00.20.40.60.81.0Accuracy Valid Test 2px 4px 6px 8px10px 12px 14px Intensity of Shift0.00.20.40.60.81.01.21.41.6Brier (b) Translated MNIST 0.0 0.2 0.4 0.6 0.8 1.0 ¿0.20.30.40.50.60.70.8Accuracy on examples p(yjx)¸¿Vanilla Temp Scaling Ensemble LL-Dropout SVI LL-SVI Dropout (c) Confidence vs Acc Rotated 60 0.0 0.2 0.4 0.6 0.8 1.0 ¿0200040006000800010000Number of examples p(yjx)¸¿ Vanilla Temp Scaling Ensemble LL-Dropout SVI LL-SVI Dropout (d) Count vs Confidence Rotated 60 0.0 0.5 1.0 1.5 2.0 2.5 Entropy (Nats)050001000015000200002500030000# of ExamplesVanilla SVI LL-Dropout LL-SVI Ensemble Temp Scaling Dropout(e) Entropy on OOD 0.0 0.2 0.4 0.6 0.8 1.0 ¿0200040006000800010000Number of examples p(yjx)¸¿ Vanilla Temp Scaling Ensemble LL-Dropout SVI LL-SVI Dropout (f) Confidence on OOD Figure 1: Results on MNIST: 1(a) and 1(b) show accuracy and Brier score as the data is increasingly shifted. Shaded regions represent standard error over 10 runs. To understand the discrepancy between accuracy and Brier score, we explore the predictive distributions of each method by looking at the confidence of the predictions in 1(c) and 1(d). We also explore the entropy and confidence of each method on entirely OOD data in 1(e) and 1(f). SVI has lower accuracy on the validation and test splits, but it is significantly more robust to dataset shift as evidenced by a lower Brier score, lower overall confidence 1(d) and higher predictive entropy under shift (1(c)) and OOD data (1(e),1(f)). the behavior of the predictive distributions of the models. In addition, we predict on a completely OOD dataset, Not-MNIST (Bulatov, 2011), and observe the entropy of the model’s predictions. We summarize some of our findings in Figure 1 and discuss below. What we would like to see: Naturally, we expect the accuracy of a model to degrade as it predicts on increasingly shifted data, and ideally this reduction in accuracy would coincide with increased forecaster entropy. A model that was well-calibrated on the training and validation distributions would ideally remain so on shifted data. If calibration (ECE or Brier reliability) remained as consistent as possible, practitioners and downstream tasks could take into account that a model is becoming increasingly uncertain. On the completely OOD data, one would expect the predictive distributions to be of high entropy. Essentially, we would like the predictions to indicate that a model “knows what it does not know” due to the inputs straying away from the training data distribution. What we observe: We see in Figures 1(a) and 1(b) that accuracy certainly degrades as a function of shift for all methods tested, and they are difficult to disambiguate on that metric. However, the Brier score paints a clearer picture and we see a significant difference between methods, i.e. prediction quality degrades more significantly for some methods than others. An important observation is that while calibrating on the validation set leads to well-calibrated predictions on the test set, it does not guarantee calibration on shifted data . In fact, nearly all other methods (except vanilla) perform better than the state-of-the-art post-hoc calibration (Temperature scaling) in terms of Brier score under shift. While SVI achieves the worst accuracy on the test set, it actually outperforms all other methods by a much larger margin when exposed to significant shift. In Figures 1(c) and 1(d) we look at the distribution of confidences for each method to understand the discrepancy between metrics. We see in Figure 1(d) that SVI has the lowest confidence in general but in Figure 1(c) we observe that SVI gives the highest accuracy at high confidence (or conversely is much less frequently confidently wrong), which can be important for high-stakes applications. Most methods demonstrate very low entropy (Figure 1(e)) and give high confidence predictions (Figure 1(f)) on data that is entirely OOD, i.e. they are confidently wrong about completely OOD data. 5Test 1 2 3 4 5 Shift intensity0.20.30.40.50.60.70.80.91.0Accuracy Method Vanilla Tem p Scaling Ensemble DropoutLL Dropout SVI LL SVI Test 1 2 3 4 5 Shift intensity0.00.10.20.30.40.50.60.70.8Accuracy Method Vanilla Tem p Scaling EnsembleDropout LL Dropout LL SVI Test 1 2 3 4 5 Shift intensity0.000.050.100.150.200.250.300.35ECE Method Vanilla Tem p Scaling EnsembleDropout LL Dropout LL SVITest 1 2 3 4 5 Shift intensity0.00.10.20.30.40.50.60.7ECE Method Vanilla Tem p Scaling Ensemble DropoutLL Dropout SVI LL SVI(a) CIFAR-10 (b) ImageNetFigure 2: Calibration under distributional shift: a detailed comparison of accuracy and ECE under all types of corruptions on (a) CIFAR-10 and (b) ImageNet. For each method we show the mean on the test set and summarize the results on each intensity of shift with a box plot. Each box shows the quartiles summarizing the results across all (16) types of shift while the error bars indicate the min and max across different shift types. Figures showing additional metrics are provided in Figures S4 (CIFAR-10) and S5 (ImageNet). Tables for numerical comparisons are provided in Appendix G. 4.2 Image Models: CIFAR-10 and ImageNet We now study the predictive distributions of residual networks (He et al., 2016) trained on two benchmark image datasets, CIFAR-10 (Krizhevsky, 2009) and ImageNet (Deng et al., 2009), under distributional shift. We use 20-layer and 50-layer ResNets for CIFAR-10 and ImageNet respectively. For shifted data we use 80 different distortions (16 different types with 5 levels of intensity each, see Appendix B for illustrations) introduced by Hendrycks & Dietterich (2019). To evaluate predictions of CIFAR-10 models on entirely OOD data, we use the SVHN dataset (Netzer et al., 2011). Figure 2 summarizes the accuracy and ECE for CIFAR-10 (top) and ImageNet (bottom) across all 80 combinations of corruptions and intensities from (Hendrycks & Dietterich, 2019). Figure 3 inspects the predictive distributions of the models on CIFAR-10 (top) and ImageNet (bottom) for shifted (Gaussian blur) and OOD data. Classifiers on both datasets show poorer accuracy and calibration with increasing shift. Comparing accuracy for different methods, we see that ensembles achieve 60.0 0.2 0.4 0.6 0.8 1.0 ¿0.650.700.750.800.850.900.951.00Accuracy on examples p(yjx)¸¿ Vanilla Temp Scaling Ensemble Dropout LL-Dropout SVI LL-SVI(a) CIFAR: Confidence vs Accuracy 0.0 0.2 0.4 0.6 0.8 1.0 ¿10002000300040005000600070008000900010000Number of examples p(yjx)¸¿ Vanilla Temp Scaling Ensemble Dropout LL-Dropout SVI LL-SVI (b) CIFAR: Count vs Confidence 0.0 0.5 1.0 1.5 2.0 2.5 Entropy (Nats)0100020003000400050006000700080009000# of ExamplesVanilla Dropout SVI LL-Dropout LL-SVI Ensemble Temp Scaling (c) CIFAR: Entropy on OOD 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 ¿0.30.40.50.60.70.80.91.0Accuracy on examples p(yjx)¸¿ Vanilla LL SVI Dropout LL Dropout Ensemble Temp Scaling (d) ImageNet: Confidence vs Acc 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 ¿01000020000300004000050000Number of examples p(yjx)¸¿Vanilla LL SVI Dropout LL Dropout Ensemble Temp Scaling (e) ImageNet: Count vs Confidence 0.0 0.2 0.4 0.6 0.8 1.0 ¿0200040006000800010000Number of examples p(yjx)¸¿ Vanilla Temp Scaling Ensemble Dropout LL-Dropout SVI LL-SVI (f) CIFAR: Confidence on OOD Figure 3: Results on CIFAR-10 and ImageNet. Left column: 3(a) and 3(d) show accuracy as a function of confidence. Middle column: 3(b) and 3(e) show the number of examples greater than given confidence values for Gaussian blur of intensity 3. Right column: 3(c) and 3(f) show histogram of entropy and confidences from CIFAR-trained models on a completely different dataset (SVHN). highest accuracy under distributional shift. Comparing the ECE for different methods, we observe that while the methods achieve comparable low values of ECE for small values of shift, ensembles outperform the other methods for larger values of shift. To test whether this result is due simply to the larger aggregate capacity of the ensemble, we trained models with double the number of filters for the Vanilla and Dropout methods. The higher-capacity models showed no better accuracy or calibration for medium- to high-shift than the corresponding lower-capacity models (see Appendix C). In Figures S8 and S9 we also explore the effect of the number of samples used in dropout, SVI and last layer methods and size of the ensemble, on CIFAR-10. We found that while increasing ensemble size up to 50 did help, most of the gains of ensembling could be achieved with only 5 models. Interestingly, while temperature scaling achieves low ECE for low values of shift, the ECE increases significantly as the shift increases, which indicates that calibration on the i.i.d. validation dataset does not guarantee calibration under distributional shift . (Note that for ImageNet, we found similar trends considering just the top-5 predicted classes, See Figure S5.) Furthermore, the results show that while temperature scaling helps significantly over the vanilla method, ensembles and dropout tend to be better. In Figure 3, we see that ensembles and dropout are more accurate at higher confidence. However, in 3(c) we see that temperature scaling gives the highest entropy on OOD data. Ensembles consistently have high accuracy but also high entropy on OOD data. We refer to Appendix C for additional results; Figures S4 and S5 report additional metrics on CIFAR-10 and ImageNet, such as Brier score (and its component terms), as well as top-5 error for increasing values of shift. Overall, ensembles consistently perform best across metrics and dropout consistently performed better than temperature scaling and last layer methods. While the relative ordering of methods is consistent on both CIFAR-10 and ImageNet (ensembles perform best), the ordering is quite different from that on MNIST where SVI performs best. Interestingly, LL-SVI and LL-Dropout perform worse than the vanilla method on shifted datasets as well as SVHN. We also evaluate a variational Gaussian process as a last layer method in Appendix E but it did not outperform LL-SVI and LL-Dropout. 4.3 Text Models Following Hendrycks & Gimpel (2017), we train an LSTM (Hochreiter & Schmidhuber, 1997) on the 20newsgroups dataset (Lang, 1995) and assess the model’s robustness under distributional shift 70.00.51.01.52.02.5 Entropy0.00.51.01.52.02.53.03.5DensityVanilla In-dist. Skewed OOD −0.50.00.51.01.52.02.5 Entropy0.00.51.01.52.02.53.03.5LL-SVI 0.00.51.01.52.02.5 Entropy0.00.51.01.52.02.53.03.5Dropout 0.00.51.01.52.02.5 Entropy0.00.51.01.52.02.53.03.5LL-Dropout 0.00.51.01.52.02.5 Entropy0.00.51.01.52.02.53.03.5Ensemble 0.0 0.5 1.0 1.5 2.0 2.5 Entropy0.00.51.01.52.02.53.03.5Temp Scaling 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 ¿405060708090100Accuracy on examples p(yjx)¸¿ Vanilla LL-SVI Dropout LL-Dropout Ensemble Temp Scaling(a) Confidence vs Acc. 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 ¿020004000600080001000012000Number of examples p(yjx)¸¿ Vanilla LL-SVI Dropout LL-Dropout Ensemble Temp Scaling (b) Confidence vs Count 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 ¿405060708090100Accuracy on examples p(yjx)¸¿ Vanilla LL-SVI Dropout LL-Dropout Ensemble Temp Scaling (c) Confidence vs Accuracy 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 ¿020004000600080001000012000Number of examples p(yjx)¸¿ Vanilla LL-SVI Dropout LL-Dropout Ensemble Temp Scaling (d) Confidence vs Count Figure 4: Top row: Histograms of the entropy of the predictive distributions for in-distribution (solid lines), shifted (dotted lines), and completely different OOD (dashed lines) text examples. Bottom row: Confidence score vs accuracy and count respectively when evaluated for in-distribution and in-distribution shift text examples (a,b), and in-distribution and OOD text examples (c,d). and OOD text. We use the even-numbered classes (10 classes out of 20) as in-distribution and the 10 odd-numbered classes as shifted data. We provide additional details in Appendix A.4. We look at confidence vs accuracy when the test data consists of a mix of in-distribution and either shifted or completely OOD data, in this case the One Billion Word Benchmark (LM1B) (Chelba et al., 2013). Figure 4 (bottom row) shows the results. Ensembles significantly outperform all other methods, and achieve better trade-off between accuracy versus confidence. Surprisingly, LL-Dropout and LL-SVI perform worse than the vanilla method, giving higher confidence incorrect predictions, especially when tested on fully OOD data. Figure 4 reports histograms of predictive entropy on in-distribution data and compares them to those for the shifted and OOD datasets. This reflects how amenable each method is to abstaining from prediction by applying a threshold on the entropy. As expected, most methods achieve the highest predictive entropy on the completely OOD dataset, followed by the shifted dataset and then the in-distribution test dataset. Only ensembles have consistently higher entropy on the shifted data, which explains why they perform best on the confidence vs accuracy curves in the second row of Figure 4. Compared with the vanilla model, Dropout and LL-SVI have more a distinct separation between in-distribution and shifted or OOD data. While Dropout and LL-Dropout perform similarly on in-distribution, LL-Dropout exhibits less uncertainty than Dropout on shifted and OOD data. Temperature scaling does not appear to increase uncertainty significantly on the shifted data. 4.4 Ad-Click Model with Categorical Features Finally, we evaluate the performance of different methods on the Criteo Display Advertising Chal- lenge6dataset, a binary classification task consisting of 37M examples with 13 numerical and 26 categorical features per example. We introduce shift by reassigning each categorical feature to a random new token with some fixed probability that controls the intensity of shift. This coarsely simulates a type of shift observed in non-stationary categorical features as category tokens appear and disappear over time, for example due to hash collisions. The model consists of a 3-hidden-layer multi-layer-perceptron (MLP) with hashed and embedded categorical features and achieves a negative log-likelihood of approximately 0.5 (contest winners achieved 0.44). Due to class imbalance ( 25% of examples are positive), we report AUC instead of classification accuracy. Results from these experiments are depicted in Figure 5. (Figure S7 in Appendix C shows additional results including ECE and Brier score decomposition.) We observe that ensembles are superior in terms of both AUC and Brier score for most of the values of shift, with the performance gap between ensembles and other methods generally increasing as the shift increases. Both Dropout model variants yielded improved AUC on shifted data, and Dropout surpassed ensembles in Brier 6https://www.kaggle.com/c/criteo-display-ad-challenge 8Train Valid Test 5% 15% 25% 35% 45% 55% 65% 75% 85% 95%0.550.600.650.700.750.80AUC Train Valid Test 5% 15% 25% 35% 45% 55% 65% 75% 85% 95%0.320.340.360.380.400.420.440.46Brier ScoreVanilla Dropout LL-Dropout LL-SVI Temp Scaling Ensemble 0.4 0.5 0.6 0.7 0.8 0.9 1.0 ¿01000020000300004000050000Number of examples p(yjx)¸¿ Vanilla Dropout LL-Dropout LL-SVI Temp Scaling Ensemble 0.4 0.5 0.6 0.7 0.8 0.9 1.0 ¿0.700.750.800.850.900.951.00Accuracy on examples p(yjx)¸¿Vanilla Dropout LL-Dropout LL-SVI Temp Scaling EnsembleFigure 5: Results on Criteo: The first two plots show degrading AUCs and Brier scores with increasing shift while the latter two depict the distribution of prediction confidences and their corresponding accuracies at 75% randomization of categorical features. SVI is excluded as it performed too poorly. score at shift-randomization values above 60%. SVI proved challenging to train, and the resulting model uniformly performed poorly; LL-SVI fared better but generally did not improve upon the vanilla model. Strikingly, temperature scaling has a worse Brier score than Vanilla indicating that post-hoc calibration on the validation set actually harms calibration under dataset shift. 5 Takeaways and Recommendations We presented a large-scale evaluation of different methods for quantifying predictive uncertainty under dataset shift, across different data modalities and architectures. Our take-home messages are the following: Along with accuracy, the quality of uncertainty consistently degrades with increasing dataset shift regardless of method. Better calibration and accuracy on the i.i.d. test dataset does not usually translate to better calibration under dataset shift (shifted versions as well as completely different OOD data). Post-hoc calibration (on i.i.d validation) with temperature scaling leads to well-calibrated uncer- tainty on the i.i.d. test set and small values of shift, but is significantly outperformed by methods that take epistemic uncertainty into account as the shift increases. Last layer Dropout exhibits less uncertainty on shifted and OOD datasets than Dropout. SVI is very promising on MNIST/CIFAR but it is difficult to get to work on larger datasets such as ImageNet and other architectures such as LSTMs. The relative ordering of methods is mostly consistent (except for MNIST) across our experiments. The relative ordering of methods on MNIST is not reflective of their ordering on other datasets. Deep ensembles seem to perform the best across most metrics and be more robust to dataset shift. We found that relatively small ensemble size (e.g. M= 5) may be sufficient (Appendix D). We also compared the set of methods on a real-world challenging genomics problem from Ren et al. (2019). Our observations were consistent with the other experiments in the paper. Deep ensembles performed best, but there remains significant room for improvement, as with the other experiments in the paper. See Section F for details. We hope that this benchmark is useful to the community and inspires more research on uncertainty under dataset shift, which seems challenging for existing methods. While we focused only on the quality of predictive uncertainty, applications may also need to consider computational and memory costs of the methods; Table S1 in Appendix A.9 discusses these costs, and the best performing methods tend to be more expensive. Reducing the computational and memory costs, while retaining the same performance under dataset shift, would also be a key research challenge. Acknowledgements We thank Alexander D’Amour, Jakub ´Swia ¸tkowski and our reviewers for helpful feedback that improved the manuscript. 9References Alemi, A. A., Fischer, I., and Dillon, J. V . Uncertainty in the variational information bottleneck. arXiv preprint arXiv:1807.00906 , 2018. Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., and Man ´e, D. Concrete problems in AI safety. arXiv preprint arXiv:1606.06565 , 2016. Behrmann, J., Duvenaud, D., and Jacobsen, J.-H. Invertible residual networks. arXiv preprint arXiv:1811.00995 , 2018. Bishop, C. M. Novelty Detection and Neural Network Validation. IEE Proceedings-Vision, Image and Signal processing , 141(4):217–222, 1994. Blundell, C., Cornebise, J., Kavukcuoglu, K., and Wierstra, D. Weight uncertainty in neural networks. InICML , 2015. Bojarski, M., Testa, D. D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., Zhang, X., Zhao, J., and Zieba, K. End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316 , 2016. Brier, G. W. Verification of forecasts expressed in terms of probability. Monthly weather review , 1950. Br¨ocker, J. Reliability, sufficiency, and the decomposition of proper scores. Quarterly Journal of the Royal Meteorological Society , 135(643):1512–1519, 2009. Bulatov, Y . NotMNIST dataset, 2011. URL http://yaroslavvb.blogspot.com/2011/09/notmnist-dataset.html . Chelba, C., Mikolov, T., Schuster, M., Ge, Q., Brants, T., Koehn, P., and Robinson, T. One billion word benchmark for measuring progress in statistical language modeling. arXiv preprint arXiv:1312.3005 , 2013. DeGroot, M. H. and Fienberg, S. E. The comparison and evaluation of forecasters. The statistician , 1983. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. ImageNet: A Large-Scale Hierarchical Image Database. In Computer Vision and Pattern Recognition , 2009. Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., and Thrun, S. Dermatologist- level classification of skin cancer with deep neural networks. Nature , 542, 1 2017. Gal, Y . and Ghahramani, Z. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In ICML , 2016. Geifman, Y . and El-Yaniv, R. Selective classification for deep neural networks. In NeurIPS , 2017. Gneiting, T. and Raftery, A. E. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association , 102(477):359–378, 2007. Golovin, D., Solnik, B., Moitra, S., Kochanski, G., Karro, J., and Sculley, D. Google vizier: A service for black-box optimization. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pp. 1487–1495. ACM, 2017. Graves, A. Practical variational inference for neural networks. In NeurIPS , 2011. Guo, C., Pleiss, G., Sun, Y ., and Weinberger, K. Q. On calibration of modern neural networks. In International Conference on Machine Learning , 2017. He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pp. 770–778, 2016. Hendrycks, D. and Dietterich, T. Benchmarking neural network robustness to common corruptions and perturbations. In ICLR , 2019. 10Hendrycks, D. and Gimpel, K. A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks. In ICLR , 2017. Hensman, J., Matthews, A., and Ghahramani, Z. Scalable variational gaussian process classification. InInternational Conference on Artificial Intelligence and Statistics . JMLR, 2015. Hern ´andez-Lobato, J. M. and Adams, R. Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks. In ICML , 2015. Hochreiter, S. and Schmidhuber, J. Long short-term memory. Neural Comput. , 9(8):1735–1780, November 1997. Kendall, A. and Gal, Y . What uncertainties do we need in Bayesian deep learning for computer vision? In NeurIPS , 2017. Kingma, D. and Ba, J. Adam: A Method for Stochastic Optimization. In ICLR , 2014. Kingma, D. P., Mohamed, S., Rezende, D. J., and Welling, M. Semi-supervised learning with deep generative models. In NeurIPS , 2014. Kingma, D. P., Salimans, T., and Welling, M. Variational dropout and the local reparameterization trick. In NeurIPS , 2015. Klambauer, G., Unterthiner, T., Mayr, A., and Hochreiter, S. Self-normalizing neural networks. In NeurIPS , 2017. Krizhevsky, A. Learning multiple layers of features from tiny images. 2009. Lakshminarayanan, B., Pritzel, A., and Blundell, C. Simple and Scalable Predictive Uncertainty Estimation Using Deep Ensembles. In NeurIPS , 2017. Lang, K. Newsweeder: Learning to filter netnews. In Machine Learning . 1995. LeCun, Y ., Bottou, L., Bengio, Y ., and Haffner, P. Gradient-based learning applied to document recognition. In Proceedings of the IEEE , November 1998. Lee, K., Lee, K., Lee, H., and Shin, J. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In NeurIPS , 2018. Liang, S., Li, Y ., and Srikant, R. Enhancing the Reliability of Out-of-Distribution Image Detection in Neural Networks. ICLR , 2018. Lipton, Z. C. and Steinhardt, J. Troubling trends in machine learning scholarship. arXiv preprint arXiv:1807.03341 , 2018. Louizos, C. and Welling, M. Structured and efficient variational deep learning with matrix Gaussian posteriors. arXiv preprint arXiv:1603.04733 , 2016. Louizos, C. and Welling, M. Multiplicative Normalizing Flows for Variational Bayesian Neural Networks. In ICML , 2017. MacKay, D. J. Bayesian methods for adaptive models . PhD thesis, California Institute of Technology, 1992. MacKay, D. J. and Gibbs, M. N. Density Networks. Statistics and Neural Networks: Advances at the Interface , 1999. Naeini, M. P., Cooper, G. F., and Hauskrecht, M. Obtaining Well Calibrated Probabilities Using Bayesian Binning. In AAAI , pp. 2901–2907, 2015. Nalisnick, E., Matsukawa, A., Teh, Y . W., Gorur, D., and Lakshminarayanan, B. Hybrid models with deep and invertible features. arXiv preprint arXiv:1902.02767 , 2019. Netzer, Y ., Wang, T., Coates, A., Bissacco, A., Wu, B., and Ng, A. Y . Reading Digits in Natural Images with Unsupervised Feature Learning. In NeurIPS Workshop on Deep Learning and Unsupervised Feature Learning , 2011. 11Osband, I., Blundell, C., Pritzel, A., and Van Roy, B. Deep exploration via bootstrapped DQN. In NeurIPS , 2016. Platt, J. C. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In Advances in Large Margin Classifiers , pp. 61–74. MIT Press, 1999. Quinonero-Candela, J., Rasmussen, C. E., Sinz, F., Bousquet, O., and Sch ¨olkopf, B. Evaluating predictive uncertainty challenge. In Machine Learning Challenges . Springer, 2006. Rahimi, A. and Recht, B. An addendum to alchemy, 2017. Ren, J., Liu, P. J., Fertig, E., Snoek, J., Poplin, R., DePristo, M. A., Dillon, J. V ., and Lakshmi- narayanan, B. Likelihood ratios for out-of-distribution detection. arXiv preprint arXiv:1906.02845 , 2019. Riquelme, C., Tucker, G., and Snoek, J. Deep Bayesian Bandits Showdown: An Empirical Compari- son of Bayesian Deep Networks for Thompson Sampling. In ICLR , 2018. Sculley, D., Snoek, J., Wiltschko, A., and Rahimi, A. Winner’s curse? On pace, progress, and empirical rigor. 2018. Shafaei, A., Schmidt, M., and Little, J. J. Does Your Model Know the Digit 6 Is Not a Cat? A Less Biased Evaluation of “Outlier” Detectors. ArXiv e-Print arXiv:1809.04729 , 2018. Srivastava, R. K., Greff, K., and Schmidhuber, J. Training Very Deep Networks. In NeurIPS , 2015. Sugiyama, M., Lawrence, N. D., Schwaighofer, A., et al. Dataset shift in machine learning . The MIT Press, 2009. Welling, M. and Teh, Y . W. Bayesian Learning via Stochastic Gradient Langevin Dynamics. In ICML , 2011. Wen, Y ., Vicol, P., Ba, J., Tran, D., and Grosse, R. Flipout: Efficient pseudo-independent weight perturbations on mini-batches. arXiv preprint arXiv:1803.04386 , 2018. Wu, A., Nowozin, S., Meeds, E., Turner, R. E., Hernandez-Lobato, J. M., and Gaunt, A. L. Determin- istic Variational Inference for Robust Bayesian Neural Networks. In ICLR , 2019. 12Can You Trust Your Model’s Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift: Appendix A Model Details A.1 MNIST We evaluated both LeNet and a fully-connected neural network (MLP) under shift on MNIST. We observed similar trends across metrics for both models, so we report results only for LeNet in Section 4.1. LeNet and MLP were trained for 20 epochs using the Adam optimizer (Kingma & Ba, 2014) and used ReLU activation functions. For stochastic methods, we averaged 300 sample predictions to yield a predictive distribution, and the ensemble model used 10 instances trained from independent random initializations. The MLP architecture consists of two hidden layers of 200 units each with dropout applied before every dense layer. The LeNet architecture (LeCun et al., 1998) applies two convolutional layers 3x3 kernels of 32 and 64 filters respectively) followed by two fully-connected layers with one hidden layer of 128 activations; dropout was applied before each fully-connected layer. We employed hyperparameter tuning (See Section A.8) to select the training batch size, learning rate, and dropout rate. A.2 CIFAR-10 Our CIFAR model used the ResNet-20 V1 architecture with ReLU activations. Model parameters were trained for 200 epochs using the Adam optimizer and employed a learning rate schedule that multiplied an initial learning rate by 0.1, 0.01, 0.001, and 0.0005 at steps 80, 120, 160, and 180 respectively. Training inputs were randomly distorted using horizontal flips and random crops preceded by 4-pixel padding as described in (He et al., 2016). For relevant methods, dropout was applied before each convolutional and dense layer (excluding the raw inputs), and stochastic methods sampled 128 predictions per sample. Hyperparameter tuning was used to select the initial learning rate, training batch size, and the dropout rate. A.3 ImageNet 2012 Our ImageNet model used the ResNet-50 V1 architecture with ReLU activations and was trained for 90 epochs using SGD with Nesterov momentum. The learning rate schedule linearly ramps up to a base rate in 5 epochs and scales down by a factor of 10 at each of epochs 30, 60, and 80. As with the CIFAR-10 model, stochastic methods used a sample-size of 128. Training images were distorted with random horizontal flips and random crops. A.4 20 Newsgroups We use a pre-processing strategy similar to the one proposed by Hendrycks & Gimpel (2017) for 20 Newsgroups. We build a vocabulary of size 30,000 words and words are indexed based on the word frequencies. The rare words are encoded as unknown words. We fix the length of each text input by setting a limit of 250 words, and those longer than 250 words are truncated, and those shorter than 250 words are padded with zeros. Text in even-numbered classes are used as in-distribution inputs, and text from the odd-numbered of classes are used shifted OOD inputs. A dataset with the same number of randomly selected text inputs from the LM1B dataset (Chelba et al., 2013) is used as completely different OOD dataset. The classifier is trained and evaluated only using the text from the even-numbered in-distribution classes in the training dataset. The final test results are evaluated based on in-distribution test dataset, shift OOD test dataset, and LM1B dataset. The vanilla model uses a one-layer LSTM model of size 32 and a dense layer to predict the 10 class probabilities based on word embedding of size 128. A dropout rate of 0.1 is applied to both the LSTM layer and the dense layer for the Dropout model. The LL-SVI model replaces the last dense layer with a Bayesian layer, the ensemble model aggregates 10 vanilla models, and stochastic methods sample 5 predictions per example. The vanilla model accuracy for in-distribution test data is 0.955. 13A.5 Criteo Each categorical feature xkfrom the Criteo dataset was encoded by hashing the string token into a fixed number of buckets Nkand either encoding the hash-bin as a one-hot vector if Nk<110or embedding each bucket as a dkdimensional vector otherwise. This dense feature vector, concatenated with 13 numerical features, feeds into a batch-norm layer followed by a 3-hidden-layer MLP. Each model was trained for one epoch using the Adam optimizer with a non-decaying learning rate. Values ofNkanddkwere tuned to maximize log-likelihood for a vanilla model, and the resulting architectural parameters were applied to all methods. This tuning yielded hidden-layers of size 2572, 1454, and 1596, and hash-bucket counts and embedding dimensions of sizes listed below: Nk= [1373;2148;4847;9781;396;28;3591;2798;14;7403;2511;5598;9501; 46;4753;4056;23;3828;5856;12;4226;23;61;3098;494;5087] dk= [3;9;29;11;17;0;14;4;0;12;19;24;29;0;13;25;0;8;29;0;22;0;0;31;0;29] Learning rate, batch size, and dropout rate were further tuned for each method. Stochastic methods used 128 prediction samples per example. A.6 Stochastic Variational Inference Details For MNIST we used Flipout (Wen et al., 2018), where we replaced each dense layer and convolutional layer with mean-field variational dense and convolutional Flipout layers respectively. Variational inference for deep ResNets (He et al., 2016) is non-trivial, so for CIFAR we replaced a single linear layer per residual branch with a Flipout layer, removed batch normalization, added Selu non-linearities (Klambauer et al., 2017), empirical Bayes for the prior standard deviations as in Wu et al. (2019) and careful tuning of the initialization via Bayesian optimization. A.7 Variational Gaussian Process Details For the experiments where Gaussian Processes were compared, we used Variational Gaussian Processes to fit the model logits as in Hensman et al. (2015). These were then passed through a Categorical distribution and numerically integrated over using Gauss-Hermite quadrature. Each class was treated as a separate Gaussian Process, with 100 inducing points used for each class. The inducing points were initialized with model outputs on random dataset examples for CIFAR, and with Gaussian noise for MNIST. Uniform noise inducing point initialization was also tested but there was negligible difference between the three methods. All zero inducing points initializations numerically failed early on in training. Exponentiated quadratic plus linear kernels were used for all experiments. 250 samples were drawn from the logit distribution during training time to get a better estimate of the ELBO to backpropagate through. 250 logit samples were drawn at test time. 105Iwas added to the diagonal of the covariance matrix to ensure positive definiteness. We used 100 trials of random hyperparamter settings, selecting the configuration with the best final validation accuracy. The learning rate was tuned in [104;1:0]on a log scale; the initial kernel amplitude in [2:0;2:0]; the initial kernel length scale in [2:0;2:0]; the variational distribution covariance was initialized to sIwhereswas tuned in [0:1;2:0];11in Adam was tuned on [102;0:15]on a log scale. The Adam optimizer with a batch size of 512 was used, training for the same number of epochs as other methods. The same learning rate schedule was as other methods for the model and kernel parameters, but the learning rate for the variational parameters also included a 5 epoch warmup in order to help with numerical stability. A.8 Hyperparameter Tuning Hyperparameters were optimized through Bayesian optimization using Google Vizier (Golovin et al., 2017). We maximized the log-likelihood on a validation set that was held out from training (10K examples for MNIST and CIFAR-10, 125K examples for ImageNet). We optimized log-likelihood rather than accuracy since the former is a proper scoring rule. 14A.9 Computational and Memory Complexity of Different methods In addition to performance, applications may also need to consider computational and memory costs; Table S1 discusses them for each method. Table S1: Computational and memory costs for evaluated methods. Notation: mrepresents flops or storage for the full model, drepresents flops or storage for the last layer, kdenotes replications, z the number of inducing points for Gaussian Processes, ndenotes number of evaluated points, and v denotes the validation set size. Serving/training compute is identical except that v= 0for serving. Implicit in this table is a memory/compute tradeoff for sampling. Sampled weights/masks need not be stored explicitly via PRNG seed reuse; we assume the computational cost of sampling is zero. Method Compute/ n Storage Vanillam m Temp Scaling m+vm=n m LL-Dropout m+d(k1)m LL-SVIm+d(k1)m+d SVImk 2m Dropoutmk m Gaussian Process m+z3m+z2 Ensemble mk mk B Shifted Images We distorted MNIST images using rotations with spline filter interpolation and cyclic translations as depicted in Figure S1. For the corrupted ImageNet dataset, we used ImageNet-C (Hendrycks & Dietterich, 2019). Figure S2 shows examples of ImageNet-C images at varying corruption intensities. Figure S3 shows ImageNet- C images with the 16 corruptions analyzed in this paper, at intensity 3 (on a scale of 1 to 5). (a) Rotations (b) Cyclic translations Figure S1: Examples of rotated and cyclically translated MNIST digits. Results for accuracy and calibration on rotated/translated MNIST are shown in Figure 1. 15Figure S2: Examples of ImageNet images corrupted by Gaussian blur, at intensities of 0 (uncorrupted image) through 5 (maximum corruption included in ImageNet-C). Figure S3: Examples of 16 corruption types in ImageNet-C images, at corruption intensity 3 (on a scale from 1–5). The same corruptions were applied to CIFAR-10. Figure 2 and Section C show boxplots for each uncertainty method and corruption intensity, spanning all corruption types. 16C Evaluating uncertainty under distributional shift: Additional Results Figures S4, S5 and S7 show comprehensive results on CIFAR-10, ImageNet and Criteo respectively across various metrics including Brier score, along with the components of the Brier score : reliability (lower means better calibration) and resolution (higher values indicate better predictive quality). Ensembles and dropout outperform all other methods across corruptions, while LL SVI shows no improvement over the baseline model. Figure S6 shows accuracy and ECE for models with double the number of ResNet filters; the higher-capacity models are not better calibrated than their lower- capacity counterparts, suggesting that the good calibration performance of ensembles is not due simply to higher capacity. 17Test 1 2 3 4 5 Shift intensity0.20.30.40.50.60.70.80.91.0AccuracyMethod Vanilla Temp Scaling Ensemble DropoutLL Dropout SVI LL SVI Test 1 2 3 4 5 Shift intensity0.00.10.20.30.40.50.60.7ECEMethod Vanilla Temp Scaling Ensemble DropoutLL Dropout SVI LL SVI Test 1 2 3 4 5 Shift intensity0.00.20.40.60.81.01.21.4Brier ScoreMethod Vanilla Temp Scaling Ensemble DropoutLL Dropout SVI LL SVI Test 1 2 3 4 5 Shift intensity1.01.11.21.31.41.51.6Brier ReliabilityMethod Vanilla Temp Scaling Ensemble DropoutLL Dropout SVI LL SVI Test 1 2 3 4 5 Shift intensity1.01.11.21.31.41.51.61.71.8Brier ResolutionMethod Vanilla Temp Scaling Ensemble DropoutLL Dropout SVI LL SVIFigure S4: Boxplots facilitating comparison of methods for each shift level showing detailed compar- isons of various metrics under all types of corruptions on CIFAR-10. Each box shows the quartiles summarizing the results across all types of shift while the error bars indicate the min and max across different shift types. 18Test 1 2 3 4 5 Shift intensity0.00.10.20.30.40.50.60.70.8AccuracyMethod Vanilla Temp Scaling EnsembleDropout LL Dropout LL SVI Test 1 2 3 4 5 Shift intensity0.00.20.40.60.81.0Accuracy Top 5Method Vanilla Temp Scaling EnsembleDropout LL Dropout LL SVI Test 1 2 3 4 5 Shift intensity0.000.050.100.150.200.250.300.35ECEMethod Vanilla Temp Scaling EnsembleDropout LL Dropout LL SVI Test 1 2 3 4 5 Shift intensity0.000.020.040.060.080.100.12ECE Top 5Method Vanilla Temp Scaling EnsembleDropout LL Dropout LL SVI Test 1 2 3 4 5 Shift intensity024681012Negative Log LikelihoodMethod Vanilla Temp Scaling EnsembleDropout LL Dropout LL SVI Test 1 2 3 4 5 Shift intensity0.40.60.81.01.2Brier ScoreMethod Vanilla Temp Scaling EnsembleDropout LL Dropout LL SVIFigure S5: Boxplots facilitating comparison of methods for each shift level showing detailed compar- isons of various metrics under all types of corruptions on ImageNet. Each box shows the quartiles summarizing the results across all types of shift while the error bars indicate the min and max across different shift types. 19Test 1 2 3 4 5 Shift intensity0.10.20.30.40.50.60.70.80.91.0AccuracyMethod Wide Dropout Vanilla EnsembleDropout Wide Vanilla Test 1 2 3 4 5 Shift intensity0.00.10.20.30.40.50.60.7ECEMethod Wide Dropout Vanilla EnsembleDropout Wide VanillaFigure S6: Boxplots facilitating comparison of results for higher-capacity models (’Wide Vanilla’ and ’Wide Dropout’) with their lower-capacity counterparts on CIFAR. Each box shows the quartiles summarizing the results across all types of shift while the error bars indicate the min and max across different shift types. 20Train Valid Test 5% 15% 25% 35% 45% 55% 65% 75% 85% 95%0.550.600.650.700.750.80AUC Train Valid Test 5% 15% 25% 35% 45% 55% 65% 75% 85% 95%0.320.340.360.380.400.420.440.46Brier ScoreVanilla Dropout LL-Dropout LL-SVI Temp Scaling Ensemble Train Valid Test 5% 15% 25% 35% 45% 55% 65% 75% 85% 95%0.2350.2400.2450.2500.2550.2600.2650.270Brier ResolutionVanilla Dropout LL-Dropout LL-SVI Temp Scaling Ensemble Train Valid Test 5% 15% 25% 35% 45% 55% 65% 75% 85% 95%0.020.030.040.050.060.070.080.09Brier Reliability Train Valid Test 5% 15% 25% 35% 45% 55% 65% 75% 85% 95%0.000.050.100.150.20ECEFigure S7: Comprehensive comparison of metrics on Criteo models. The Brier decomposition reveals that the majority of its degradation is due to worsening reliability, and this component alone appears to largely explain the ranking of methods in total Brier score. Ensemble notably degrades most rapidly in resolution but persists with better reliability compared other methods for most of the data-corruption range; on ECE it remains roughly in the middle among explored methods. Dropout (and to a lesser extend LL-Dropout) perform best on ECE and experience slower degradation in both resolution and reliability leading it to surpass ensembles at the severe range of data corruption. Total Brier score and AUC results are discussed in detail in Section 4.4. D Effect of the number of samples on the quality of uncertainty Figure S8 shows the effect of the number of sample sizes used by Dropout, SVI (and last-layer variants) on the quality of predictive uncertainty, as measured by the Brier score. Increasing the number of samples has little effect on last-layer variants, whereas increasing the number of samples improves the performance for SVI and Dropout, with diminishing returns beyond size 5. Train Valid Test 1 2 3 4 50.00.10.20.30.40.50.60.70.80.9Brier score (a) Dropout Train Valid Test 1 2 3 4 50.00.20.40.60.81.01.2Brier score (b) LL-Dropout Train Valid Test 1 2 3 4 50.00.10.20.30.40.50.60.70.80.9Brier score (c) SVI Train Valid Test 1 2 3 4 50.00.20.40.60.81.0Brier score1 2 4 8 16 32 64 (d) LL-SVI Figure S8: Effect of Dropout and SVI sample sizes on CIFAR-10 Brier scores under increasing Gaussian blur. See Section 4.2 for full results on CIFAR-10. Figure S9 shows the effect of ensemble size on CIFAR-10 (top) and ImageNet (bottom). Similar to SVI and Dropout, we see that increasing the number of models in the ensemble improves perfor- mance with diminishing returns beyond size 5. As mentioned earlier, the Brier score can be further 21decomposed into BS = calibration + renement = reliability + uncertainty resolution where reliability#measures calibration as the average violation of long-term true label frequencies, and renement = uncertainty resolution , where uncertainty is the marginal uncertainty over labels (independent of predictions) and resolution"measures the deviation of individual predictions from the marginal. Train Valid Test 1 2 3 4 50.00.20.40.60.81.01.2Brier score1 2 5 10 25 50 (a) Brier Score Train Valid Test 1 2 3 4 50.000.050.100.150.200.250.300.350.400.45Brier reliability1 2 5 10 25 50 (b) Brier Reliability Train Valid Test 1 2 3 4 50.00.10.20.30.40.50.60.70.80.9Brier resolution1 2 5 10 25 50 (c) Brier Resolution Train Valid Test 1 2 3 4 50.10.20.30.40.50.60.70.80.91.0Brier score1 2 4 8 (d) Brier Score Train Valid Test 1 2 3 4 50.040.060.080.100.120.140.160.18Brier reliability1 2 4 8 (e) Brier Reliability Train Valid Test 1 2 3 4 50.00.10.20.30.40.50.60.70.80.9Brier resolution1 2 4 8 (f) Brier Resolution Figure S9: Effect of the ensemble size on CIFAR-10 (top row) and ImageNet (bottom row) Brier scores under increasing Gaussian-blur shift. We additionally show the Brier score components: Reliability (lower means better calibration) and Resolution (higher values indicate better predictive quality). Note that the scales for Reliability are significantly smaller than the other plots. E Variational Gaussian Process Results 220 1 2 3 4 5 Skew intensity0.20.40.60.81.01.21.4Brier ScoreMethod Baseline EnsembleVariational GP(a) Brier Score 0 1 2 3 4 5 6kew intenVity0.20.30.40.50.60.70.80.9Accuracy0ethoG BaVeline EnVePEle9ariational G3 (b) Accuracy 0 1 2 3 4 5 Skew intensity0.10.20.30.40.50.6ECEMethod Baseline EnsembleVariational GP (c) ECE Figure S10: Uncertainty metrics across shift levels on CIFAR-10, where level 0 is the test set, using a last layer Variational Gaussian Process. See Appendix A.7 for experiment details. 23F OOD detection for genomic sequences We studied the set of methods for detecting OOD genomic sequence, as a challenging realistic problem for OOD detection proposed by Ren et al. (2019). Classifiers are trained on 10 in-distribution bacteria classes, and tested for OOD detection of 60 OOD bacteria classes. The model architecture is the same as that in Ren et al. (2019): a convolutional neural networks with 1000 filters of length 20, followed by a global max pooling layer, a dense layer of 1000 units, and a last dense layer that outputs class prediction logits. For the dropout method, we add a dropout layer each after the max pooling layer and the dense layer respectively. For the LL-Dropout method, only a dropout layer after the dense layer is added. We use the dropout rate of 0.2. For the LL-SVI method, we replace the last dense layer with a stochastic variational inference dense layer. The classification accuracy for in-distribution is around 0.8 for the various types of classifiers. Figure S11 shows the confidence vs (a) accuracy and (b) count when the test data consists of a mix of in-distribution and OOD data. Ensembles significantly outperform all other methods, and achieve better trade-off between accuracy versus confidence. Dropout performs better than Temp Scaling, and they both perform better than LL-Dropout, LL-SVI, and the Vanilla method. Note that the accuracy on examples p(yjx)0:9for the best method is still below 65%, suggesting that this realistic genomic sequences dataset is a challenging problem to benchmark future methods. 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 ¿404550556065Accuracy on examples p(yjx)¸¿ Vanilla Temp Scaling Ensemble LL-SVI LL-Dropout Dropout (a) Confidence vs Accuracy 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 ¿60008000100001200014000160001800020000Number of examples p(yjx)¸¿ Vanilla Temp Scaling Ensemble LL-SVI LL-Dropout Dropout (b) Confidence vs Count Figure S11: Confidence score vs accuracy and count respectively when evaluated for in-distribution and OOD genomic sequences. G Tables of Metrics The tables below report quartiles of Brier score, negative log-likelihood, and ECE for each model and dataset where quartiles are computed over all corrupted variants of the dataset. G.1 CIFAR-10 Method Vanilla Temp. Scaling Ensembles Dropout LL-Dropout SVI LL-SVI Brier Score (25th) 0.243 0.227 0.165 0.215 0.259 0.250 0.246 Brier Score (50th) 0.425 0.392 0.299 0.349 0.416 0.363 0.431 Brier Score (75th) 0.747 0.670 0.572 0.633 0.728 0.604 0.732 NLL (25th) 2.356 1.685 1.543 1.684 2.275 1.628 2.352 NLL (50th) 1.120 0.871 0.653 0.771 1.086 0.823 1.158 NLL (75th) 0.578 0.473 0.342 0.446 0.626 0.533 0.591 ECE (25th) 0.057 0.022 0.031 0.021 0.069 0.029 0.058 ECE (50th) 0.127 0.049 0.037 0.034 0.136 0.064 0.135 ECE (75th) 0.288 0.180 0.110 0.174 0.292 0.187 0.275 24G.2 ImageNet Method Vanilla Temp. Scaling Ensembles Dropout LL-Dropout LL-SVI Brier Score (25th) 0.553 0.551 0.503 0.577 0.550 0.590 Brier Score (50th) 0.733 0.726 0.667 0.754 0.723 0.766 Brier Score (75th) 0.914 0.899 0.835 0.922 0.896 0.938 NLL (25th) 1.859 1.848 1.621 1.957 1.830 2.218 NLL (50th) 2.912 2.837 2.446 3.046 2.858 3.504 NLL (75th) 4.305 4.186 3.661 4.567 4.208 5.199 ECE (25th) 0.057 0.031 0.022 0.017 0.034 0.065 ECE (50th) 0.102 0.072 0.032 0.043 0.071 0.106 ECE (75th) 0.164 0.129 0.053 0.109 0.123 0.148 G.3 Criteo Method Vanilla Temp. Scaling Ensembles Dropout LL-Dropout SVI LL-SVI Brier Score (25th) 0.353 0.355 0.336 0.350 0.353 0.512 0.361 Brier Score (50th) 0.385 0.391 0.366 0.373 0.379 0.512 0.396 Brier Score (75th) 0.409 0.416 0.395 0.393 0.403 0.512 0.421 NLL (25th) 0.581 0.594 0.508 0.532 0.542 7.479 0.554 NLL (50th) 0.788 0.829 0.552 0.577 0.600 7.479 0.633 NLL (75th) 0.986 1.047 0.608 0.624 0.664 7.479 0.711 ECE (25th) 0.041 0.055 0.044 0.043 0.052 0.254 0.066 ECE (50th) 0.097 0.113 0.100 0.085 0.100 0.254 0.127 ECE (75th) 0.135 0.149 0.141 0.116 0.136 0.254 0.162 25

You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you s... You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you select really corresponds to the excerpt: there will be details mentioned in the excerpt that should appear in the paper. If you read an abstract and it seems like it could be the paper we’re looking for, read the paper to make sure. Also: sometimes you’ll read a paper that cites the paper we’re looking for. In such cases, please go to the references in order to find the full name of the paper we’re looking for, and search for it, and then select it. For this task, we use the Comparative Question Completion dataset introduced by [CITATION]. This consists of questions in which one of a pair of coordinated elements is masked; the target is the masked phrase You are now given an excerpt. Find me the paper cited in the excerpt, using the tools described above. Please make sure that the paper you select really corresponds to the excerpt: there will be details mentioned in the excerpt that should appear in the paper. If you read an abstract and it seems like it could be the paper we’re looking for, read the paper to make sure. Also: sometimes you’ll read a paper that cites the paper we’re looking for. In such cases, please go to the references in order to find the full name of the paper we’re looking for, and search for it, and then select it. For this task, we use the Comparative Question Completion dataset introduced by [CITATION]. This consists of questions in which one of a pair of coordinated elements is masked; the target is the masked phrase

- Paper ID: dc230ae929bdee9e8c9c811d62ea1f6725012cda Title: SC-Ques: A Sentence Completion Question Dataset for English as a Second Langu... - Paper ID: dc230ae929bdee9e8c9c811d62ea1f6725012cda Title: SC-Ques: A Sentence Completion Question Dataset for English as a Second Language Learners Abstract: Sentence completion (SC) questions present a sentence with one or more blanks that need to be filled in, three to five possible words or phrases as options. SC questions are widely used for students learning English as a Second Language (ESL). In this paper, we present a large-scale SC dataset, \textsc{SC-Ques}, which is made up of 289,148 ESL SC questions from real-world standardized English examinations. Furthermore, we build a comprehensive benchmark of automatically solving the SC questions by training the large-scale pre-trained language models on the proposed \textsc{SC-Ques} dataset. We conduct detailed analysis of the baseline models performance, limitations and trade-offs. The data and our code are available for research purposes from: \url{https://github.com/ai4ed/SC-Ques}. Citation Count: 0 - Paper ID: 35d15... - Paper ID: dc230ae929bdee9e8c9c811d62ea1f6725012cda Title: SC-Ques: A Sentence Completion Question Dataset for English as a Second Language Learners Abstract: Sentence completion (SC) questions present a sentence with one or more blanks that need to be filled in, three to five possible words or phrases as options. SC questions are widely used for students learning English as a Second Language (ESL). In this paper, we present a large-scale SC dataset, \textsc{SC-Ques}, which is made up of 289,148 ESL SC questions from real-world standardized English examinations. Furthermore, we build a comprehensive benchmark of automatically solving the SC questions by training the large-scale pre-trained language models on the proposed \textsc{SC-Ques} dataset. We conduct detailed analysis of the baseline models performance, limitations and trade-offs. The data and our code are available for research purposes from: \url{https://github.com/ai4ed/SC-Ques}. Citation Count: 0 - Paper ID: 35d153e0d675d0054eb2b7673f84e3c0aafe5bf2 Title: AssistQ: Affordance-centric Question-driven Task Completion for Egocentric Assistant Abstract: A long-standing goal of intelligent assistants such as AR glasses/robots has been to assist users in affordance-centric real-world scenarios, such as"how can I run the microwave for 1 minute?". However, there is still no clear task definition and suitable benchmarks. In this paper, we define a new task called Affordance-centric Question-driven Task Completion, where the AI assistant should learn from instructional videos to provide step-by-step help in the user's view. To support the task, we constructed AssistQ, a new dataset comprising 531 question-answer samples from 100 newly filmed instructional videos. We also developed a novel Question-to-Actions (Q2A) model to address the AQTC task and validate it on the AssistQ dataset. The results show that our model significantly outperforms several VQA-related baselines while still having large room for improvement. We expect our task and dataset to advance Egocentric AI Assistant's development. Our project page is available at: https://showlab.github.io/assistq/. Citation Count: 17 - Paper ID: f57afb6c8addfc7a32f9be5916a374a542d1a026 Title: ECG-QA: A Comprehensive Question Answering Dataset Combined With Electrocardiogram Abstract: Question answering (QA) in the field of healthcare has received much attention due to significant advancements in natural language processing. However, existing healthcare QA datasets primarily focus on medical images, clinical notes, or structured electronic health record tables. This leaves the vast potential of combining electrocardiogram (ECG) data with these systems largely untapped. To address this gap, we present ECG-QA, the first QA dataset specifically designed for ECG analysis. The dataset comprises a total of 70 question templates that cover a wide range of clinically relevant ECG topics, each validated by an ECG expert to ensure their clinical utility. As a result, our dataset includes diverse ECG interpretation questions, including those that require a comparative analysis of two different ECGs. In addition, we have conducted numerous experiments to provide valuable insights for future research directions. We believe that ECG-QA will serve as a valuable resource for the development of intelligent QA systems capable of assisting clinicians in ECG interpretations. Dataset URL: https://github.com/Jwoo5/ecg-qa Citation Count: 3 - Paper ID: 63d637283e7807d2a72ce6d94ccb35c9edd76fed Title: QUADRo: Dataset and Models for QUestion-Answer Database Retrieval Abstract: An effective paradigm for building Automated Question Answering systems is the re-use of previously answered questions, e.g., for FAQs or forum applications. Given a database (DB) of question/answer (q/a) pairs, it is possible to answer a target question by scanning the DB for similar questions. In this paper, we scale this approach to open domain, making it competitive with other standard methods, e.g., unstructured document or graph based. For this purpose, we (i) build a large scale DB of 6.3M q/a pairs, using public questions, (ii) design a new system based on neural IR and a q/a pair reranker, and (iii) construct training and test data to perform comparative experiments with our models. We demonstrate that Transformer-based models using (q,a) pairs outperform models only based on question representation, for both neural search and reranking. Additionally, we show that our DB-based approach is competitive with Web-based methods, i.e., a QA system built on top the BING search engine, demonstrating the challenge of finding relevant information. Finally, we make our data and models available for future research. Citation Count: 1 - Paper ID: bd1af42e4099d9661bdbaed160bcc4bb65c086e2 Title: A Question of Style: A Dataset for Analyzing Formality on Different Levels Abstract: Accounting for different degrees of formality is crucial for producing contextually appropriate language. To assist NLP applications concerned with this problem and formality analysis in general, we present the first dataset of sentences from a wide range of genres assessed on a continuous informal-formal scale via comparative judgments. It is the first corpus with a comprehensive perspective on German sentence-level formality overall. We compare machine learning models for formality scoring, a task we treat as a regression problem, on our dataset. Finally, we investigate the relation between sentence- and document-level formality and evaluate leveraging sentence-based annotations for assessing formality on documents. Citation Count: 1 - Paper ID: 15187df6d7b3c9cc8036b2fc5f57fee188ec1344 Title: Efficient Arabic Query Auto-Completion for Question Answering at a University Abstract: In this paper we describe an implementation of an Arabic query auto-completion system for student question-answering at a University. University students make many inquiries concerning academic life: about majors, concentrations, dates, instructors, courses, rooms, exams and more. Auto-completion (AC) has recently been part of many user interfaces, such as search bars on web pages, social media sites and mobile applications. We investigate multiple approaches to completion candidate generation and ranking and the role Arabic NLP may play in that. After experimenting with other options, we collected the data used in our system directly from students at the University. This data can be expanded to account for more types of queries and ways to express information needs. We describe our dataset, give an evaluation of individual system components and of the system results in general. The goal of this work is to improve the answer search experience by reducing the time of entering the query and biasing the completed query towards unambiguous and easily answerable questions. We divided the implementation into several stages and report on the results for each stage separately. Despite the difficulties of Arabic NLP, the results of our system were encouraging and compared well with other QAC systems described in the literature. Finally, we give an informal evaluation of overall results and the improvement resulting from using QAC for our QA system. Citation Count: 0 - Paper ID: 5a69c2e26a221f8fddeffab7d579dc2cdfca6e33 Title: A Comparative Study of Transformers Embeddings for Question Answering in Arabic Private Documents Abstract: Question answering (QA) tasks in natural language processing (NLP) are tricky, particularly when used with Arabic private documents. This is due to the complexity of Arabic language and the lack of sufficient annotated datasets for Arabic QA. Recent research findings have demonstrated the effectiveness of transformer embeddings for English QA. Their efficiency for QA in Arabic private documents, however, has not been fully investigated. In this study, various transformer embeddings for QA in Arabic private documents are compared. The effectiveness of these embeddings is evaluated by assessing a golden dataset extracted from private documents. The results show that the accuracy of QA in Arabic private documents can be significantly improved by combining transformer embeddings with count vectorization. Furthermore, different transformer embeddings have distinct strengths and weaknesses. Citation Count: 0 - Paper ID: 6d40db49cec2a543e01a4ef651f053ae935274fc Title: Sequence-to-Sequence Knowledge Graph Completion and Question Answering Abstract: Knowledge graph embedding (KGE) models represent each entity and relation of a knowledge graph (KG) with low-dimensional embedding vectors. These methods have recently been applied to KG link prediction and question answering over incomplete KGs (KGQA). KGEs typically create an embedding for each entity in the graph, which results in large model sizes on real-world graphs with millions of entities. For downstream tasks these atomic entity representations often need to be integrated into a multi stage pipeline, limiting their utility. We show that an off-the-shelf encoder-decoder Transformer model can serve as a scalable and versatile KGE model obtaining state-of-the-art results for KG link prediction and incomplete KG question answering. We achieve this by posing KG link prediction as a sequence-to-sequence task and exchange the triple scoring approach taken by prior KGE methods with autoregressive decoding. Such a simple but powerful method reduces the model size up to 98% compared to conventional KGE models while keeping inference time tractable. After finetuning this model on the task of KGQA over incomplete KGs, our approach outperforms baselines on multiple large-scale datasets without extensive hyperparameter tuning. Citation Count: 70 - Paper ID: 8e01f640c39f974bddbcca957d1580f0dd47f511 Title: Joint Knowledge Graph Completion and Question Answering Abstract: Knowledge graph reasoning plays a pivotal role in many real-world applications, such as network alignment, computational fact-checking, recommendation, and many more. Among these applications, knowledge graph completion (KGC) and multi-hop question answering over knowledge graph (Multi-hop KGQA) are two representative reasoning tasks. In the vast majority of the existing works, the two tasks are considered separately with different models or algorithms. However, we envision that KGC and Multi-hop KGQA are closely related to each other. Therefore, the two tasks will benefit from each other if they are approached adequately. In this work, we propose a neural model named BiNet to jointly handle KGC and multi-hop KGQA, and formulate it as a multi-task learning problem. Specifically, our proposed model leverages a shared embedding space and an answer scoring module, which allows the two tasks to automatically share latent features and learn the interactions between natural language question decoder and answer scoring module. Compared to the existing methods, the proposed BiNet model addresses both multi-hop KGQA and KGC tasks simultaneously with superior performance. Experiment results show that BiNet outperforms state-of-the-art methods on a wide range of KGQA and KGC benchmark datasets. Citation Count: 25 - Paper ID: 5a8dd8b644049fdfc877cb15e1c4a9b249ea1c5c Title: Template-based Question Answering analysis on the LC-QuAD2.0 Dataset Abstract: In recent years, template-based question answer has picked up steam as a solution for evaluating RDF triples. Once we delve into the domain of template-based question answering, two important questions arise which are, the size of the dataset used as the knowledge base and the process of training used on that knowledge base. Previous studies attempted this problem with the LC-QuAD dataset and recursive neural network for training. This paper studies the same problem with a larger and newer benchmark dataset called LC-QuAD 2.0 and training using different machine learning models. The objective of this paper is to provide a comparative study using the newer LC-QuAD 2.0 dataset that has an updated schema and 30,000 question-answer pairs. Our study will focus on using and comparing two Machine Learning models and 3 different pre-processing techniques to generate results and identify the best model for this problem. Citation Count: 3

- Paper ID: f588cf92ccc0b73741e766fe370e461e7cf21c6a Title: Sources of Evidence for Interactive Table Completion Abstract: An importan... - Paper ID: f588cf92ccc0b73741e766fe370e461e7cf21c6a Title: Sources of Evidence for Interactive Table Completion Abstract: An important question in interactive information retrieval (IIR) is: How can we support searchers with specific types of search tasks? We describe an auxiliary support tool referred to as the "Matrix''. The Matrix tool was designed to support searchers with comparative search tasks, which require comparing items along different dimensions. The Matrix was designed as a grid of rows and columns representing the items and dimensions related to a comparative task. The Matrix was integrated with a custom-built search interface, which allowed users to search for information and drag-and-drop relevant passages directly into cells in the Matrix. We investigate the following general question: Given a partially completed Matrix, can a system automatically populate empty cells in the Matrix with relevant passages? To this end, we conducted two crowdsource... - Paper ID: f588cf92ccc0b73741e766fe370e461e7cf21c6a Title: Sources of Evidence for Interactive Table Completion Abstract: An important question in interactive information retrieval (IIR) is: How can we support searchers with specific types of search tasks? We describe an auxiliary support tool referred to as the "Matrix''. The Matrix tool was designed to support searchers with comparative search tasks, which require comparing items along different dimensions. The Matrix was designed as a grid of rows and columns representing the items and dimensions related to a comparative task. The Matrix was integrated with a custom-built search interface, which allowed users to search for information and drag-and-drop relevant passages directly into cells in the Matrix. We investigate the following general question: Given a partially completed Matrix, can a system automatically populate empty cells in the Matrix with relevant passages? To this end, we conducted two crowdsourced studies in which participants were assigned comparative tasks and asked to use our system (integrated search interface + Matrix) to populate every cell in the Matrix. After gathering this data, we evaluated machine-learned models for ranking passages in response to an empty Matrix cell and partially completed Matrix. We address two research questions: (RQ1) What are useful types of features for this predictive task? and (RQ2) How does performance vary based on the level of Matrix completion? We view our research as a step towards designing support tools that: (1) help users organize information while searching and (2) can autocomplete search tasks by exploiting the task structure and a searcher's partial solution. Citation Count: 2 - Paper ID: e4b3eb206109f7bae0bf0e96381a63d91edc0b56 Title: Visual Encodings of Temporal Uncertainty: A Comparative User Study Abstract: A number of studies have investigated different ways of visualizing uncertainty. However, in the temporal dimension, it is still an open question how to best represent uncertainty, since the special characteristics of time require special visual encodings and may provoke different interpretations. Thus, we have conducted a comprehensive study comparing alternative visual encodings of intervals with uncertain start and end times: gradient plots, violin plots, accumulated probability plots, error bars, centered error bars, and ambiguation. Our results reveal significant differences in error rates and completion time for these different visualization types and different tasks. We recommend using ambiguation - using a lighter color value to represent uncertain regions - or error bars for judging durations and temporal bounds, and gradient plots - using fading color or transparency - for judging probability values. Citation Count: 55 - Paper ID: a07e176ca3d5d97503a6d9d1de794c2e8f707024 Title: Randomized comparative study of child and caregiver responses to three software functions added to the Japanese version of the electronic Pediatric Quality of Life Inventory (ePedsQL) questionnaire Abstract: None Citation Count: 1 - Paper ID: 33df8056529a6dff2bbb5cf05056036e02193d0f Title: Comparative Analysis of Usability of Data Entry Design Patterns for Mobile Applications Abstract: None Citation Count: 6 - Paper ID: ff64aada7af34e26c8f3b93b52e474cb3e79c1bd Title: Do tutors make a difference in online learning? A comparative study in two Open Online Courses Abstract: Two free fully online courses were offered by Peoples-uni on its Open Online Courses site, both as self-paced courses available any time and as courses run over four weeks with tutor-led discussions. We tested the hypothesis that there are no measurable differences in outcomes between the two delivery methods. Similar numbers attended both versions of each course; students came from multiple countries and backgrounds. Numbers of discussion forum posts were greater in tutor-led than self-paced courses. Measured outcomes of certificates of completion, quiz completion and marks gained were very similar and not statistically significantly different between the tutor-led and the self-paced versions of either course. In light of little discernible difference in outcome between self-paced learning compared with courses including tutor-led discussions, the utility of the time cost to tutors is in question. The findings may be relevant to others designing online courses, including MOOCs. Citation Count: 5 - Paper ID: f5803a67afb5b8ca73b97caa79ebb1bdef7103c6 Title: A comparative study on two XML editors (oxygon and ultraedit) Abstract: Extensible Markup Language (XML) is a simple, universal format supported by the W3C designed to data representation, exchange and transition on the web between different applications. XML editor is a markup language editor that provides a platform with functional features to facilitate XML editing. There is a variety of XML editors available in the market. Amongst the editors, the two most popular XML editors are Oxygen XML Editor and UltraEdit. Both of them offer easy functionalities such as syntax validation, auto-completion, multiple tabs, advanced search and replacement tools. However, novice programmers find difficult to choose suitable editors. For this reason, this paper aims to compare the features of Oxygen XML Editor and UltraEdit so that the users are aware of their capability and capacity, leading them to select an editor that can fulfill their requirements. This paper begins discussions on the objectives of the research, research question, literature review, research methodology and findings. Citation Count: 1 - Paper ID: 350a764c56020f2cb10c71f64268a72777e7927f Title: Comparison Analysis of Breadth First Search and Depth Limited Search Algorithms in Sudoku Game Abstract: Sudoku is a game that sharpens the brain and is very well known. But the problem faced in this condition is how we can find a solution for the completion of this game. Problems with the Sudoku game can be solved by using the concept of Artificial Intelligence (AI). Some of the algorithms that can be used are the Breadth First Search (BFS) and Depth Limited Search (DLS) algorithms. The purpose of this research is to find a solution for Sudoku and make a comparative analysis of the search results of the two algorithms. The results obtained are application design in the form of a simulation of the completion of the Sudoku game problem with two algorithms. And it has been proven from the two algorithms that DLS is more efficient and faster than BFS. While BFS itself has advantages, in terms of a more structured and systematic search system that is able to find all possible numbers in each box. In this case, if a Sudoku question has more than one answer, then BFS will find it. Citation Count: 17 - Paper ID: 40e65db9be0fd9cc5321b1e68fa176b196b4db6b Title: A comparison of visual symptoms after reading text from computer screen and hard copy. Abstract: Purpose:  The basic purpose of this study was to evaluate the association of asthenopic symptoms with prolonged computer use or sustained near work and its comparison with hardcopy (printed material) and to find out the change in accommodation and convergence due to its association with asthenopic symptoms. METHOD : A comparative cross sectional study was conducted in emetropes. This study consists of 30 young participants who performed test on computer screen and hard copy for consistent 40 minutes. And the working distance maintained was 30cm. After completion of test they were asked to fill self-designed proforma with question regarding their visual discomfort after performing test from computer screen and hard copy. RESULTS: Final results indicated that overall percentage of change in visual symptoms were greater in computer users than hard copy. 40% participants reported blurring of vision and headache from computer and 30% from hard copy. And 63.33% participants felt eyestrain and irritation in eyes from computer screen and 36.67% from hard copy.53.33% felt discomfort in eyes from computer screen and 40% from hard copy. Results obtained through paired sample t-test revealed that there was a significant change in p-values of NPC and NPA. P-value of NPC obtained after reading from soft copy was.001 and p-value of NPA was .003 while the p-value of NPC obtained after reading from hard copy was .002 and .040 p-value of NPA was observed. CONCLUSION: Results demonstrated that accommodation fatigue has a marked impact on visual discomfort while doing near work. But higher percentage of asthenopic symptoms was observed during computer screen. And it shows that even use of modern VDT or computer screen still causes visual discomfort as shown in results. So it is comparatively easy to read from hard copy than from computer screen. Citation Count: 0 - Paper ID: 6d40db49cec2a543e01a4ef651f053ae935274fc Title: Sequence-to-Sequence Knowledge Graph Completion and Question Answering Abstract: Knowledge graph embedding (KGE) models represent each entity and relation of a knowledge graph (KG) with low-dimensional embedding vectors. These methods have recently been applied to KG link prediction and question answering over incomplete KGs (KGQA). KGEs typically create an embedding for each entity in the graph, which results in large model sizes on real-world graphs with millions of entities. For downstream tasks these atomic entity representations often need to be integrated into a multi stage pipeline, limiting their utility. We show that an off-the-shelf encoder-decoder Transformer model can serve as a scalable and versatile KGE model obtaining state-of-the-art results for KG link prediction and incomplete KG question answering. We achieve this by posing KG link prediction as a sequence-to-sequence task and exchange the triple scoring approach taken by prior KGE methods with autoregressive decoding. Such a simple but powerful method reduces the model size up to 98% compared to conventional KGE models while keeping inference time tractable. After finetuning this model on the task of KGQA over incomplete KGs, our approach outperforms baselines on multiple large-scale datasets without extensive hyperparameter tuning. Citation Count: 70 - Paper ID: 300bb9a145d70a2ce858a525461cb8ff0d222cdb Title: QTC4SO: Automatic Question Title Completion for Stack Overflow Abstract: Question posts with low-quality titles often discourage potential answerers in Stack Overflow. In previous studies, researchers mainly focused on directly generating question titles by analyzing the contents of the posts. However, the quality of the generated titles is still limited by the information available in the post contents. A more effective way is to provide accurate completion suggestions when developers compose titles. Inspired by this idea, we are the first to study the problem of automatic question title completion for Stack Overflow and then propose a novel approach QTC4SO. Specifically, we first preprocess the gathered post titles to form incomplete titles (i.e., tip information provided by developers) for simulating the scene of this task. Then we construct the multi-modal input by concatenating the incomplete title with the post’s contents (i.e., the problem description and the code snippet). Later, we adopt multi-task learning to the question title completion task for multiple programming languages. Finally, we adopt a pre-trained model T5 to learn the title completion patterns automatically. To evaluate the effectiveness of QTC4SO, we gathered 164,748 high-quality posts from Stack Overflow by covering eight popular programming languages. Our empirical results show that compared with the approaches of directly generating question titles, our proposed approach QTC4SO is more practical in automatic and human evaluation. Therefore, our study provides a new direction for automatic question title generation and we hope more researchers can pay attention to this problem in the future. Citation Count: 1

- Paper ID: dc230ae929bdee9e8c9c811d62ea1f6725012cda Title: SC-Ques: A Sentence Completion Question Dataset for English as a Second Langu... - Paper ID: dc230ae929bdee9e8c9c811d62ea1f6725012cda Title: SC-Ques: A Sentence Completion Question Dataset for English as a Second Language Learners Abstract: Sentence completion (SC) questions present a sentence with one or more blanks that need to be filled in, three to five possible words or phrases as options. SC questions are widely used for students learning English as a Second Language (ESL). In this paper, we present a large-scale SC dataset, \textsc{SC-Ques}, which is made up of 289,148 ESL SC questions from real-world standardized English examinations. Furthermore, we build a comprehensive benchmark of automatically solving the SC questions by training the large-scale pre-trained language models on the proposed \textsc{SC-Ques} dataset. We conduct detailed analysis of the baseline models performance, limitations and trade-offs. The data and our code are available for research purposes from: \url{https://github.com/ai4ed/SC-Ques}. Citation Count: 0 - Paper ID: 35d15... - Paper ID: dc230ae929bdee9e8c9c811d62ea1f6725012cda Title: SC-Ques: A Sentence Completion Question Dataset for English as a Second Language Learners Abstract: Sentence completion (SC) questions present a sentence with one or more blanks that need to be filled in, three to five possible words or phrases as options. SC questions are widely used for students learning English as a Second Language (ESL). In this paper, we present a large-scale SC dataset, \textsc{SC-Ques}, which is made up of 289,148 ESL SC questions from real-world standardized English examinations. Furthermore, we build a comprehensive benchmark of automatically solving the SC questions by training the large-scale pre-trained language models on the proposed \textsc{SC-Ques} dataset. We conduct detailed analysis of the baseline models performance, limitations and trade-offs. The data and our code are available for research purposes from: \url{https://github.com/ai4ed/SC-Ques}. Citation Count: 0 - Paper ID: 35d153e0d675d0054eb2b7673f84e3c0aafe5bf2 Title: AssistQ: Affordance-centric Question-driven Task Completion for Egocentric Assistant Abstract: A long-standing goal of intelligent assistants such as AR glasses/robots has been to assist users in affordance-centric real-world scenarios, such as"how can I run the microwave for 1 minute?". However, there is still no clear task definition and suitable benchmarks. In this paper, we define a new task called Affordance-centric Question-driven Task Completion, where the AI assistant should learn from instructional videos to provide step-by-step help in the user's view. To support the task, we constructed AssistQ, a new dataset comprising 531 question-answer samples from 100 newly filmed instructional videos. We also developed a novel Question-to-Actions (Q2A) model to address the AQTC task and validate it on the AssistQ dataset. The results show that our model significantly outperforms several VQA-related baselines while still having large room for improvement. We expect our task and dataset to advance Egocentric AI Assistant's development. Our project page is available at: https://showlab.github.io/assistq/. Citation Count: 17 - Paper ID: 15187df6d7b3c9cc8036b2fc5f57fee188ec1344 Title: Efficient Arabic Query Auto-Completion for Question Answering at a University Abstract: In this paper we describe an implementation of an Arabic query auto-completion system for student question-answering at a University. University students make many inquiries concerning academic life: about majors, concentrations, dates, instructors, courses, rooms, exams and more. Auto-completion (AC) has recently been part of many user interfaces, such as search bars on web pages, social media sites and mobile applications. We investigate multiple approaches to completion candidate generation and ranking and the role Arabic NLP may play in that. After experimenting with other options, we collected the data used in our system directly from students at the University. This data can be expanded to account for more types of queries and ways to express information needs. We describe our dataset, give an evaluation of individual system components and of the system results in general. The goal of this work is to improve the answer search experience by reducing the time of entering the query and biasing the completed query towards unambiguous and easily answerable questions. We divided the implementation into several stages and report on the results for each stage separately. Despite the difficulties of Arabic NLP, the results of our system were encouraging and compared well with other QAC systems described in the literature. Finally, we give an informal evaluation of overall results and the improvement resulting from using QAC for our QA system. Citation Count: 0 - Paper ID: 9342e654bcb015240bfa934587a830ab6bffa97e Title: Towards Incomplete SPARQL Query in RDF Question Answering - A Semantic Completion Approach Abstract: RDF question/answering(Q/A) system allows users to ask questions in natural language on a knowledge base represented by RDF and retrieve answers. A common problem in RDF Q/A is that existing works tend to translate a natural language question into an incomplete SPARQL query, which means that SPARQL queries may not fully understand user’s ideas. For example, some triple patterns may be missing in the question translation stage. In this poster, we first present a siamese adaptation of the Long Short-Term Memory(LSTM) network to detect whether the SPARQL query generated by the RDF Q/A system is complete. Then, for incomplete queries, we propose a Markov-based method to supplement SPARQL queries. Finally, we compare our approach with some state-of-the-art RDF Q/A systems in the benchmark dataset. Extensive experiments confirm that our method improves the precision significantly. Citation Count: 0 - Paper ID: c70cefca5c76724b27e4615259be889d9ca3aa92 Title: AQuA: An Adversarially Authored Question-Answer Dataset for Common Sense Abstract: Commonsense reasoning is a critical AI capability, but it is difficult to construct challenging datasets that test common sense. Recent neural question answering systems, based on large pre-trained models of language, have already achieved near-human-level performance on commonsense knowledge benchmarks. These systems do not possess human-level common sense, but are able to exploit limitations of the datasets to achieve human-level scores. We introduce the CODAH dataset, an adversarially-constructed evaluation dataset for testing common sense. CODAH forms a challenging extension to the recently-proposed SWAG dataset, which tests commonsense knowledge using sentence-completion questions that describe situations observed in video. To produce a more difficult dataset, we introduce a novel procedure for question acquisition in which workers author questions designed to target weaknesses of state-of-the-art neural question answering systems. Workers are rewarded for submissions that models fail to answer correctly both before and after fine-tuning (in cross-validation). We create 2.8k questions via this procedure and evaluate the performance of multiple state-of-the-art question answering systems on our dataset. We observe a significant gap between human performance, which is 95.3%, and the performance of the best baseline accuracy of 67.5% by the BERT-Large model. Citation Count: 1 - Paper ID: 67241544ae69cf2972376439e35494f6e8c26dba Title: Learning by Asking for Embodied Visual Navigation and Task Completion Abstract: The research community has shown increasing interest in designing intelligent embodied agents that can assist humans in accomplishing tasks. Despite recent progress on related vision-language benchmarks, most prior work has focused on building agents that follow instructions rather than endowing agents the ability to ask questions to actively resolve ambiguities arising naturally in embodied environments. To empower embodied agents with the ability to interact with humans, in this work, we propose an Embodied Learning-By-Asking (ELBA) model that learns when and what questions to ask to dynamically acquire additional information for completing the task. We evaluate our model on the TEACH vision-dialog navigation and task completion dataset. Experimental results show that ELBA achieves improved task performance compared to baseline models without question-answering capabilities. Citation Count: 1 - Paper ID: d777d15677b673fe02750333181ea5e1e6dadf17 Title: Can Incremental Learning help with KG Completion? Abstract: Knowledge Graphs (KGs) are a type of knowledge representation that gained a lot of attention due to their ability to store information in a structured format. This structure representation makes KGs naturally suited for search engines and NLP tasks like question-answering (QA) and task-oriented systems; however, KGs are hard to construct. While QA datasets are more available and easier to construct, they lack structural representation. This availability of QA datasets made them a rich resource for machine learning models, but these models benefit from the implicit structure in such datasets. We propose a framework to make this structure more pronounced and extract KG from QA datasets in an end-to-end manner, allowing the system to learn new knowledge in incremental learning with a human-in-the-loop (HITL) when needed. We test our framework using the SQuAD dataset and our incremental learning approach with two datasets, YAGO3-10 and FB15K237, both of which show promising results. Citation Count: 0 - Paper ID: fe351bc1bda19583a7d6b1a96a9b18f59c40b213 Title: ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshots Abstract: We present a new task and dataset, ScreenQA, for screen content understanding via question answering. The existing screen datasets are focused either on structure and component-level understanding, or on a much higher-level composite task such as navigation and task completion. We attempt to bridge the gap between these two by annotating 86K question-answer pairs over the RICO dataset in hope to benchmark the screen reading comprehension capacity. Citation Count: 7 - Paper ID: e0e0cebc428ad4525c5c89d2b6e8103e3ddc4433 Title: Solving ESL Sentence Completion Questions via Pre-trained Neural Language Models Abstract: None Citation Count: 2 - Paper ID: 21ff941841ef2ad5eb8f992394ca049f02138938 Title: Generative adversarial network-based image completion to identify abnormal locations in digital breast tomosynthesis images Abstract: Deep learning has achieved great success in image analysis and decision making in radiology. However, a large amount of annotated imaging data is needed to construct well-performing deep learning models. A particular challenge in the context of breast cancer is the number of available cases that contain cancer, given the very low prevalence of the disease in the screening population. The question arises whether normal cases, which in the context of breast cancer screening are available in abundance, can be used to train a deep learning model that identifies locations that are abnormal. In this study, we propose to achieve this goal through the generative adversarial network (GAN)-based image completion. Our hypothesis is that if a generative network has a difficulty to correctly complete a part of an image at a certain location, then such a location is likely to represent an abnormality. We test this hypothesis using a dataset of 4348 patients with digital breast tomosynthesis (DBT) imaging from our institution. We trained our model on normal only images, to be able to fill in parts of images that were artificially removed. Then, using an independent test set, at different locations in the images, we measured how difficult it was for the network to reconstruct an artificially removed patch of the image. The difficulty was measured by mean squared error (MSE) between the original removed patch and the reconstructed patch. On average, the MSE was 2.11 times higher (with standard deviation equal to 1.01) at the locations containing expert-annotated cancerous lesions than that at the locations outside those abnormal locations. Our generative approach demonstrates a great potential for using this model to aid breast cancer detection. Citation Count: 3

- Paper ID: 4ff089aa85b4d604d075d1e1488bb11114547e98 Title: SONDHAN: A Comparative Study of Two Proficiency Language Bangla-English on Qu... - Paper ID: 4ff089aa85b4d604d075d1e1488bb11114547e98 Title: SONDHAN: A Comparative Study of Two Proficiency Language Bangla-English on Question-Answer Using Attention Mechanism Abstract: Recently, breakthroughs of NLP research have improved a range of activities, most notably the Question Answering System for many languages. Since the last few years, question answering (QA) systems have grown at a breakneck pace. With the continuous development of the network, the question-and-answer method has become a way for people to get information quickly & precisely that the user will ask and with the increase in web sourcing, any information has become available to the people as the relevant data is stored in that source. LSTM has been introduced, a focus-based deep learning model for the Q&A method in this study. It matches one of the sentences in the question and answer and solves the problem of unexpected features. Using the attention mechanism in the system provides accurate a... - Paper ID: 4ff089aa85b4d604d075d1e1488bb11114547e98 Title: SONDHAN: A Comparative Study of Two Proficiency Language Bangla-English on Question-Answer Using Attention Mechanism Abstract: Recently, breakthroughs of NLP research have improved a range of activities, most notably the Question Answering System for many languages. Since the last few years, question answering (QA) systems have grown at a breakneck pace. With the continuous development of the network, the question-and-answer method has become a way for people to get information quickly & precisely that the user will ask and with the increase in web sourcing, any information has become available to the people as the relevant data is stored in that source. LSTM has been introduced, a focus-based deep learning model for the Q&A method in this study. It matches one of the sentences in the question and answer and solves the problem of unexpected features. Using the attention mechanism in the system provides accurate answers by focusing on the specific questions of the candidate. Furthermore, we have proposed an adequate knowledge addition-based framework for the Q&A method. This memory contains a nested word or character level encoder that handles problems outside the words in the dataset or some rare words. We compare both Bangla and English-based question-answer for the dataset domain based on International GK, Bangladesh GK, and Science & Technology. A Sequence to Sequence LSTM based question-and-answer system with a total number of 10,000 data has been proposed through an attention mechanism with (99.91 and 99.48) % accuracy for Bangla and English data, respectively. Overall, LSTM works perfectly for both Bengali and English and is the best Q&A model. Citation Count: 1 - Paper ID: 7ed0faa6720cd176d57badbc0455af31a03f080c Title: Towards Expert-Level Medical Question Answering with Large Language Models Abstract: Recent artificial intelligence (AI) systems have reached milestones in"grand challenges"ranging from Go to protein-folding. The capability to retrieve medical knowledge, reason over it, and answer medical questions comparably to physicians has long been viewed as one such grand challenge. Large language models (LLMs) have catalyzed significant progress in medical question answering; Med-PaLM was the first model to exceed a"passing"score in US Medical Licensing Examination (USMLE) style questions with a score of 67.2% on the MedQA dataset. However, this and other prior work suggested significant room for improvement, especially when models' answers were compared to clinicians' answers. Here we present Med-PaLM 2, which bridges these gaps by leveraging a combination of base LLM improvements (PaLM 2), medical domain finetuning, and prompting strategies including a novel ensemble refinement approach. Med-PaLM 2 scored up to 86.5% on the MedQA dataset, improving upon Med-PaLM by over 19% and setting a new state-of-the-art. We also observed performance approaching or exceeding state-of-the-art across MedMCQA, PubMedQA, and MMLU clinical topics datasets. We performed detailed human evaluations on long-form questions along multiple axes relevant to clinical applications. In pairwise comparative ranking of 1066 consumer medical questions, physicians preferred Med-PaLM 2 answers to those produced by physicians on eight of nine axes pertaining to clinical utility (p<0.001). We also observed significant improvements compared to Med-PaLM on every evaluation axis (p<0.001) on newly introduced datasets of 240 long-form"adversarial"questions to probe LLM limitations. While further studies are necessary to validate the efficacy of these models in real-world settings, these results highlight rapid progress towards physician-level performance in medical question answering. Citation Count: 256 - Paper ID: 1e94953c5b7612bcc9de73133dfb731039af1d09 Title: Machine Learning Techniques for Diabetes Classification: A Comparative Study Abstract: —In light of the growing global diabetes epidemic, there is a pressing need for enhanced diagnostic tools and methods. Enter machine learning, which, with its data-driven predictive capabilities, can serve as a powerful ally in the battle against this chronic condition. This research took advantage of the Pima Indians Diabetes Data Set, which captures diverse patient information, both diabetic and non-diabetic. Leveraging this dataset, we undertook a rigorous comparative assessment of six dominant machine learning algorithms, specifically: Support Vector Machine, Artificial Neural Networks, Decision Tree, Random Forest, Logistic Regression, and Naive Bayes. Aiming for precision, we introduced principal component analysis to the workflow, enabling strategic dimensionality reduction and thus spotlighting the most salient data features. Upon completion of our analysis, it became evident that the Random Forest algorithm stood out, achieving an exemplary accuracy rate of 98.6% when 'BP' and 'SKIN' attributes were set aside. This discovery prompts a crucial discussion: not all data attributes weigh equally in their predictive value, and a discerning approach to feature selection can significantly optimize outcomes. Concluding, this study underscores the potential and efficiency of machine learning in diabetes diagnosis. With Random Forest leading the pack in accuracy, there's a compelling case to further embed such computational techniques in healthcare diagnostics, ushering in an era of enhanced patient care. Citation Count: 1 - Paper ID: e140f00ef50c860838c4a2f1bb55458dfdc5542a Title: lncRNA–disease association prediction method based on the nearest neighbor matrix completion model Abstract: None Citation Count: 3 - Paper ID: c70cefca5c76724b27e4615259be889d9ca3aa92 Title: AQuA: An Adversarially Authored Question-Answer Dataset for Common Sense Abstract: Commonsense reasoning is a critical AI capability, but it is difficult to construct challenging datasets that test common sense. Recent neural question answering systems, based on large pre-trained models of language, have already achieved near-human-level performance on commonsense knowledge benchmarks. These systems do not possess human-level common sense, but are able to exploit limitations of the datasets to achieve human-level scores. We introduce the CODAH dataset, an adversarially-constructed evaluation dataset for testing common sense. CODAH forms a challenging extension to the recently-proposed SWAG dataset, which tests commonsense knowledge using sentence-completion questions that describe situations observed in video. To produce a more difficult dataset, we introduce a novel procedure for question acquisition in which workers author questions designed to target weaknesses of state-of-the-art neural question answering systems. Workers are rewarded for submissions that models fail to answer correctly both before and after fine-tuning (in cross-validation). We create 2.8k questions via this procedure and evaluate the performance of multiple state-of-the-art question answering systems on our dataset. We observe a significant gap between human performance, which is 95.3%, and the performance of the best baseline accuracy of 67.5% by the BERT-Large model. Citation Count: 1 - Paper ID: 9b7c581206a6458afb4faf76ae145e16f33c6bd2 Title: Generative Pre-trained Transformer for Vietnamese Community-based COVID-19 Question Answering Abstract: Recent studies have provided empirical evidence of the wide-ranging potential of Generative Pre-trained Transformer (GPT), a pretrained language model, in the field of natural language processing. GPT has been effectively employed as a decoder within state-of-the-art (SOTA) question answering systems, yielding exceptional performance across various tasks. However, the current research landscape concerning GPT's application in Vietnamese remains limited. This paper aims to address this gap by presenting an implementation of GPT-2 for community-based question answering specifically focused on COVID-19 related queries in Vietnamese. We introduce a novel approach by conducting a comparative analysis of different Transformers vs SOTA models in the community-based COVID-19 question answering dataset. The experimental findings demonstrate that the GPT-2 models exhibit highly promising outcomes, outperforming other SOTA models as well as previous community-based COVID-19 question answering models developed for Vietnamese. Citation Count: 0 - Paper ID: 20398b63519548ebe04b73918da99d491fa034aa Title: Generating Synthetic Documents for Cross-Encoder Re-Rankers: A Comparative Study of ChatGPT and Human Experts Abstract: We investigate the usefulness of generative Large Language Models (LLMs) in generating training data for cross-encoder re-rankers in a novel direction: generating synthetic documents instead of synthetic queries. We introduce a new dataset, ChatGPT-RetrievalQA, and compare the effectiveness of models fine-tuned on LLM-generated and human-generated data. Data generated with generative LLMs can be used to augment training data, especially in domains with smaller amounts of labeled data. We build ChatGPT-RetrievalQA based on an existing dataset, human ChatGPT Comparison Corpus (HC3), consisting of public question collections with human responses and answers from ChatGPT. We fine-tune a range of cross-encoder re-rankers on either human-generated or ChatGPT-generated data. Our evaluation on MS MARCO DEV, TREC DL'19, and TREC DL'20 demonstrates that cross-encoder re-ranking models trained on ChatGPT responses are statistically significantly more effective zero-shot re-rankers than those trained on human responses. In a supervised setting, the human-trained re-rankers outperform the LLM-trained re-rankers. Our novel findings suggest that generative LLMs have high potential in generating training data for neural retrieval models. Further work is needed to determine the effect of factually wrong information in the generated responses and test our findings' generalizability with open-source LLMs. We release our data, code, and cross-encoders checkpoints for future work. Citation Count: 7 - Paper ID: 2cc72838e9ad37784e1de0e64e03e903dacd6f80 Title: An Improved Point Cloud Completion Method Based on SnowflakeNet Abstract: Point cloud completion aims to complete partial point clouds captured from the real world, which is a crucial step in the pipeline of many point cloud tasks. Among the existing methods for solving this problem, SnowflakeNet is the most outstanding. However, SnowflakeNet cannot recover the detailed structure of point clouds in latent code because it uses many max-pooling operations in the encoding stage. Therefore, we propose an improved architecture to effectively acquire and preserve more detail information from input point clouds, thereby enhancing the quality of point cloud completion. Specifically, the improved lightweight DGCNN is added to the encoder to extract local features. The geometric perception block of PoinTr is introduced to extract the global features of the point cloud, which can fully model the structural information and inter-point relationships of known point clouds. The new optimizer Adan is also used in the training process to complete the partial point clouds. Comparative experiments on Completion3D and PCN datasets show that our method is better than most current point cloud completion methods. Our method has the ability to produce the entire shape with details, including but not only smooth surfaces, well-defined edges, and distinct corners. Citation Count: 0 - Paper ID: 0a36008613d67fb3aac8345f847fc4787a0d69f3 Title: PixLore: A Dataset-driven Approach to Rich Image Captioning Abstract: In the domain of vision-language integration, generating detailed image captions poses a significant challenge due to the lack of a curated and rich dataset. This study introduces PixLore, a novel method that leverages Querying Transformers through the fine-tuning of the BLIP-2 model using the LoRa method on a standard commercial GPU. Our approach, which involves training on a carefully assembled dataset from state-of-the-art Computer Vision models combined and augmented by ChatGPT, addresses the question of whether intricate image understanding can be achieved with an ensemble of smaller-scale models. Comparative evaluations against major models such as GPT-4 and Google Bard demonstrate that PixLore-2.7B, despite having considerably fewer parameters, is rated higher than the existing State-of-the-Art models in over half of the assessments. This research not only presents a groundbreaking approach but also highlights the importance of well-curated datasets in enhancing the performance of smaller models. Citation Count: 0 - Paper ID: 14006532fd8691bf074dfc62ef1a89bf3c4a37ca Title: Image Completion: Survey and Comparative Study Abstract: Image completion is an active and interesting research area in image processing and computer graphics. Restoration and retouching of damaged areas in an undetectable form is the objective of image completion techniques. Most of the recently developed video completion methods are extensions of image completion techniques to restore the damaged frames. With respect to video completion challenges and image completion future work, we survey existing methods and introduce a new classification. The methods in each category are described in detail. In the second part of the paper, we provide a comparison and evaluation study between the most recent image completion methods qualitatively as well as quantitatively. For a fair comparison, we introduced a new dataset and evaluated four available image completion methods on the same hardware. Experimental results are conducted to highlight the strengths and drawbacks of each image completion method. Citation Count: 7

- Paper ID: 5a69c2e26a221f8fddeffab7d579dc2cdfca6e33 Title: A Comparative Study of Transformers Embeddings for Question Answering in Arab... - Paper ID: 5a69c2e26a221f8fddeffab7d579dc2cdfca6e33 Title: A Comparative Study of Transformers Embeddings for Question Answering in Arabic Private Documents Abstract: Question answering (QA) tasks in natural language processing (NLP) are tricky, particularly when used with Arabic private documents. This is due to the complexity of Arabic language and the lack of sufficient annotated datasets for Arabic QA. Recent research findings have demonstrated the effectiveness of transformer embeddings for English QA. Their efficiency for QA in Arabic private documents, however, has not been fully investigated. In this study, various transformer embeddings for QA in Arabic private documents are compared. The effectiveness of these embeddings is evaluated by assessing a golden dataset extracted from private documents. The results show that the accuracy of QA in Arabic private documents can be significantly improved by combining transformer embeddings with count vectorization. Furthermore, d... - Paper ID: 5a69c2e26a221f8fddeffab7d579dc2cdfca6e33 Title: A Comparative Study of Transformers Embeddings for Question Answering in Arabic Private Documents Abstract: Question answering (QA) tasks in natural language processing (NLP) are tricky, particularly when used with Arabic private documents. This is due to the complexity of Arabic language and the lack of sufficient annotated datasets for Arabic QA. Recent research findings have demonstrated the effectiveness of transformer embeddings for English QA. Their efficiency for QA in Arabic private documents, however, has not been fully investigated. In this study, various transformer embeddings for QA in Arabic private documents are compared. The effectiveness of these embeddings is evaluated by assessing a golden dataset extracted from private documents. The results show that the accuracy of QA in Arabic private documents can be significantly improved by combining transformer embeddings with count vectorization. Furthermore, different transformer embeddings have distinct strengths and weaknesses. Citation Count: 0 - Paper ID: 15187df6d7b3c9cc8036b2fc5f57fee188ec1344 Title: Efficient Arabic Query Auto-Completion for Question Answering at a University Abstract: In this paper we describe an implementation of an Arabic query auto-completion system for student question-answering at a University. University students make many inquiries concerning academic life: about majors, concentrations, dates, instructors, courses, rooms, exams and more. Auto-completion (AC) has recently been part of many user interfaces, such as search bars on web pages, social media sites and mobile applications. We investigate multiple approaches to completion candidate generation and ranking and the role Arabic NLP may play in that. After experimenting with other options, we collected the data used in our system directly from students at the University. This data can be expanded to account for more types of queries and ways to express information needs. We describe our dataset, give an evaluation of individual system components and of the system results in general. The goal of this work is to improve the answer search experience by reducing the time of entering the query and biasing the completed query towards unambiguous and easily answerable questions. We divided the implementation into several stages and report on the results for each stage separately. Despite the difficulties of Arabic NLP, the results of our system were encouraging and compared well with other QAC systems described in the literature. Finally, we give an informal evaluation of overall results and the improvement resulting from using QAC for our QA system. Citation Count: 0 - Paper ID: f57afb6c8addfc7a32f9be5916a374a542d1a026 Title: ECG-QA: A Comprehensive Question Answering Dataset Combined With Electrocardiogram Abstract: Question answering (QA) in the field of healthcare has received much attention due to significant advancements in natural language processing. However, existing healthcare QA datasets primarily focus on medical images, clinical notes, or structured electronic health record tables. This leaves the vast potential of combining electrocardiogram (ECG) data with these systems largely untapped. To address this gap, we present ECG-QA, the first QA dataset specifically designed for ECG analysis. The dataset comprises a total of 70 question templates that cover a wide range of clinically relevant ECG topics, each validated by an ECG expert to ensure their clinical utility. As a result, our dataset includes diverse ECG interpretation questions, including those that require a comparative analysis of two different ECGs. In addition, we have conducted numerous experiments to provide valuable insights for future research directions. We believe that ECG-QA will serve as a valuable resource for the development of intelligent QA systems capable of assisting clinicians in ECG interpretations. Dataset URL: https://github.com/Jwoo5/ecg-qa Citation Count: 3 - Paper ID: 1eb7390f67107ba07c8c674839a8f9ddd4d9c8c0 Title: Comparative Analysis of Open Source and Commercial Embedding Models for Question Answering Abstract: In this industry track presentation, we will provide a comprehensive tour of the best performing embedding models for question answering, as determined by the Massive Text Embedding Benchmark1. We showcase these models while also considering solutions offered by OpenAI and Cohere, renowned for their state-of-the-art performance. Through rigorous evaluations on internal Salesforce datasets tailored for Question Answering on Knowledge articles, we compare the performance of these models using standardized metrics. Our analysis sheds light on the current state-of-the-art in question answering using embedding models across three diverse domains. We hope that this talk's outcomes will empower practitioners and researchers to make informed decisions when selecting the most suitable solution for their specific requirements. Citation Count: 1 - Paper ID: 7ed0faa6720cd176d57badbc0455af31a03f080c Title: Towards Expert-Level Medical Question Answering with Large Language Models Abstract: Recent artificial intelligence (AI) systems have reached milestones in"grand challenges"ranging from Go to protein-folding. The capability to retrieve medical knowledge, reason over it, and answer medical questions comparably to physicians has long been viewed as one such grand challenge. Large language models (LLMs) have catalyzed significant progress in medical question answering; Med-PaLM was the first model to exceed a"passing"score in US Medical Licensing Examination (USMLE) style questions with a score of 67.2% on the MedQA dataset. However, this and other prior work suggested significant room for improvement, especially when models' answers were compared to clinicians' answers. Here we present Med-PaLM 2, which bridges these gaps by leveraging a combination of base LLM improvements (PaLM 2), medical domain finetuning, and prompting strategies including a novel ensemble refinement approach. Med-PaLM 2 scored up to 86.5% on the MedQA dataset, improving upon Med-PaLM by over 19% and setting a new state-of-the-art. We also observed performance approaching or exceeding state-of-the-art across MedMCQA, PubMedQA, and MMLU clinical topics datasets. We performed detailed human evaluations on long-form questions along multiple axes relevant to clinical applications. In pairwise comparative ranking of 1066 consumer medical questions, physicians preferred Med-PaLM 2 answers to those produced by physicians on eight of nine axes pertaining to clinical utility (p<0.001). We also observed significant improvements compared to Med-PaLM on every evaluation axis (p<0.001) on newly introduced datasets of 240 long-form"adversarial"questions to probe LLM limitations. While further studies are necessary to validate the efficacy of these models in real-world settings, these results highlight rapid progress towards physician-level performance in medical question answering. Citation Count: 256 - Paper ID: 6d40db49cec2a543e01a4ef651f053ae935274fc Title: Sequence-to-Sequence Knowledge Graph Completion and Question Answering Abstract: Knowledge graph embedding (KGE) models represent each entity and relation of a knowledge graph (KG) with low-dimensional embedding vectors. These methods have recently been applied to KG link prediction and question answering over incomplete KGs (KGQA). KGEs typically create an embedding for each entity in the graph, which results in large model sizes on real-world graphs with millions of entities. For downstream tasks these atomic entity representations often need to be integrated into a multi stage pipeline, limiting their utility. We show that an off-the-shelf encoder-decoder Transformer model can serve as a scalable and versatile KGE model obtaining state-of-the-art results for KG link prediction and incomplete KG question answering. We achieve this by posing KG link prediction as a sequence-to-sequence task and exchange the triple scoring approach taken by prior KGE methods with autoregressive decoding. Such a simple but powerful method reduces the model size up to 98% compared to conventional KGE models while keeping inference time tractable. After finetuning this model on the task of KGQA over incomplete KGs, our approach outperforms baselines on multiple large-scale datasets without extensive hyperparameter tuning. Citation Count: 70 - Paper ID: 63d637283e7807d2a72ce6d94ccb35c9edd76fed Title: QUADRo: Dataset and Models for QUestion-Answer Database Retrieval Abstract: An effective paradigm for building Automated Question Answering systems is the re-use of previously answered questions, e.g., for FAQs or forum applications. Given a database (DB) of question/answer (q/a) pairs, it is possible to answer a target question by scanning the DB for similar questions. In this paper, we scale this approach to open domain, making it competitive with other standard methods, e.g., unstructured document or graph based. For this purpose, we (i) build a large scale DB of 6.3M q/a pairs, using public questions, (ii) design a new system based on neural IR and a q/a pair reranker, and (iii) construct training and test data to perform comparative experiments with our models. We demonstrate that Transformer-based models using (q,a) pairs outperform models only based on question representation, for both neural search and reranking. Additionally, we show that our DB-based approach is competitive with Web-based methods, i.e., a QA system built on top the BING search engine, demonstrating the challenge of finding relevant information. Finally, we make our data and models available for future research. Citation Count: 1 - Paper ID: 5a8dd8b644049fdfc877cb15e1c4a9b249ea1c5c Title: Template-based Question Answering analysis on the LC-QuAD2.0 Dataset Abstract: In recent years, template-based question answer has picked up steam as a solution for evaluating RDF triples. Once we delve into the domain of template-based question answering, two important questions arise which are, the size of the dataset used as the knowledge base and the process of training used on that knowledge base. Previous studies attempted this problem with the LC-QuAD dataset and recursive neural network for training. This paper studies the same problem with a larger and newer benchmark dataset called LC-QuAD 2.0 and training using different machine learning models. The objective of this paper is to provide a comparative study using the newer LC-QuAD 2.0 dataset that has an updated schema and 30,000 question-answer pairs. Our study will focus on using and comparing two Machine Learning models and 3 different pre-processing techniques to generate results and identify the best model for this problem. Citation Count: 3 - Paper ID: 8e01f640c39f974bddbcca957d1580f0dd47f511 Title: Joint Knowledge Graph Completion and Question Answering Abstract: Knowledge graph reasoning plays a pivotal role in many real-world applications, such as network alignment, computational fact-checking, recommendation, and many more. Among these applications, knowledge graph completion (KGC) and multi-hop question answering over knowledge graph (Multi-hop KGQA) are two representative reasoning tasks. In the vast majority of the existing works, the two tasks are considered separately with different models or algorithms. However, we envision that KGC and Multi-hop KGQA are closely related to each other. Therefore, the two tasks will benefit from each other if they are approached adequately. In this work, we propose a neural model named BiNet to jointly handle KGC and multi-hop KGQA, and formulate it as a multi-task learning problem. Specifically, our proposed model leverages a shared embedding space and an answer scoring module, which allows the two tasks to automatically share latent features and learn the interactions between natural language question decoder and answer scoring module. Compared to the existing methods, the proposed BiNet model addresses both multi-hop KGQA and KGC tasks simultaneously with superior performance. Experiment results show that BiNet outperforms state-of-the-art methods on a wide range of KGQA and KGC benchmark datasets. Citation Count: 25 - Paper ID: 9b7c581206a6458afb4faf76ae145e16f33c6bd2 Title: Generative Pre-trained Transformer for Vietnamese Community-based COVID-19 Question Answering Abstract: Recent studies have provided empirical evidence of the wide-ranging potential of Generative Pre-trained Transformer (GPT), a pretrained language model, in the field of natural language processing. GPT has been effectively employed as a decoder within state-of-the-art (SOTA) question answering systems, yielding exceptional performance across various tasks. However, the current research landscape concerning GPT's application in Vietnamese remains limited. This paper aims to address this gap by presenting an implementation of GPT-2 for community-based question answering specifically focused on COVID-19 related queries in Vietnamese. We introduce a novel approach by conducting a comparative analysis of different Transformers vs SOTA models in the community-based COVID-19 question answering dataset. The experimental findings demonstrate that the GPT-2 models exhibit highly promising outcomes, outperforming other SOTA models as well as previous community-based COVID-19 question answering models developed for Vietnamese. Citation Count: 0

- Paper ID: 410e4e70ec38f9a4cfd43df6dd92adc9f739ce2a Title: Functional Partitioning of Ontologies for Natural Language Query Completion i... - Paper ID: 410e4e70ec38f9a4cfd43df6dd92adc9f739ce2a Title: Functional Partitioning of Ontologies for Natural Language Query Completion in Question Answering Systems Abstract: Query completion systems are well studied in the context of information retrieval systems that handle keyword queries. However, Natural Language Interface to Databases (NLIDB) systems that focus on syntactically correct and semantically complete queries to obtain high precision answers require a fundamentally different approach to the query completion problem as opposed to IR systems. To the best of our knowledge, we are first to focus on the problem of query completion for NLIDB systems. In particular, we introduce a novel concept of functional partitioning of an ontology and then design algorithms to intelligently use the components obtained from functional partitioning to extend a state-of-the-art NLIDB system to produce accurate and semantically meaningful query completions in the absence of query logs.... - Paper ID: 410e4e70ec38f9a4cfd43df6dd92adc9f739ce2a Title: Functional Partitioning of Ontologies for Natural Language Query Completion in Question Answering Systems Abstract: Query completion systems are well studied in the context of information retrieval systems that handle keyword queries. However, Natural Language Interface to Databases (NLIDB) systems that focus on syntactically correct and semantically complete queries to obtain high precision answers require a fundamentally different approach to the query completion problem as opposed to IR systems. To the best of our knowledge, we are first to focus on the problem of query completion for NLIDB systems. In particular, we introduce a novel concept of functional partitioning of an ontology and then design algorithms to intelligently use the components obtained from functional partitioning to extend a state-of-the-art NLIDB system to produce accurate and semantically meaningful query completions in the absence of query logs. We test the proposed query completion framework on multiple benchmark datasets and demonstrate the efficacy of our technique empirically. Citation Count: 5 - Paper ID: 40e3cd767c9520171ac8ecbd2e27f3a1bf043b61 Title: Two is Better than Many? Binary Classification as an Effective Approach to Multi-Choice Question Answering Abstract: We propose a simple refactoring of multi-choice question answering (MCQA) tasks as a series of binary classifications. The MCQA task is generally performed by scoring each (question, answer) pair normalized over all the pairs, and then selecting the answer from the pair that yield the highest score. For n answer choices, this is equivalent to an n-class classification setup where only one class (true answer) is correct. We instead show that classifying (question, true answer) as positive instances and (question, false answer) as negative instances is significantly more effective across various models and datasets. We show the efficacy of our proposed approach in different tasks – abductive reasoning, commonsense question answering, science question answering, and sentence completion. Our DeBERTa binary classification model reaches the top or close to the top performance on public leaderboards for these tasks. The source code of the proposed approach is available at https://github.com/declare-lab/TEAM. Citation Count: 6 - Paper ID: d777d15677b673fe02750333181ea5e1e6dadf17 Title: Can Incremental Learning help with KG Completion? Abstract: Knowledge Graphs (KGs) are a type of knowledge representation that gained a lot of attention due to their ability to store information in a structured format. This structure representation makes KGs naturally suited for search engines and NLP tasks like question-answering (QA) and task-oriented systems; however, KGs are hard to construct. While QA datasets are more available and easier to construct, they lack structural representation. This availability of QA datasets made them a rich resource for machine learning models, but these models benefit from the implicit structure in such datasets. We propose a framework to make this structure more pronounced and extract KG from QA datasets in an end-to-end manner, allowing the system to learn new knowledge in incremental learning with a human-in-the-loop (HITL) when needed. We test our framework using the SQuAD dataset and our incremental learning approach with two datasets, YAGO3-10 and FB15K237, both of which show promising results. Citation Count: 0 - Paper ID: fe351bc1bda19583a7d6b1a96a9b18f59c40b213 Title: ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshots Abstract: We present a new task and dataset, ScreenQA, for screen content understanding via question answering. The existing screen datasets are focused either on structure and component-level understanding, or on a much higher-level composite task such as navigation and task completion. We attempt to bridge the gap between these two by annotating 86K question-answer pairs over the RICO dataset in hope to benchmark the screen reading comprehension capacity. Citation Count: 7 - Paper ID: 441068ef17c3332d4f0af6c146e37bd8748bb90e Title: A Survey on Graph Neural Networks for Knowledge Graph Completion Abstract: Knowledge Graphs are increasingly becoming popular for a variety of downstream tasks like Question Answering and Information Retrieval. However, the Knowledge Graphs are often incomplete, thus leading to poor performance. As a result, there has been a lot of interest in the task of Knowledge Base Completion. More recently, Graph Neural Networks have been used to capture structural information inherently stored in these Knowledge Graphs and have been shown to achieve SOTA performance across a variety of datasets. In this survey, we understand the various strengths and weaknesses of the proposed methodology and try to find new exciting research problems in this area that require further investigation. Citation Count: 56 - Paper ID: 781402fae3a7a44b90855eeba8e96a872347fd4b Title: Soft Marginal TransE for Scholarly Knowledge Graph Completion Abstract: Knowledge graphs (KGs), i.e. representation of information as a semantic graph, provide a significant test bed for many tasks including question answering, recommendation, and link prediction. Various amount of scholarly metadata have been made vailable as knowledge graphs from the diversity of data providers and agents. However, these high-quantities of data remain far from quality criteria in terms of completeness while growing at a rapid pace. Most of the attempts in completing such KGs are following traditional data digitization, harvesting and collaborative curation approaches. Whereas, advanced AI-related approaches such as embedding models - specifically designed for such tasks - are usually evaluated for standard benchmarks such as Freebase and Wordnet. The tailored nature of such datasets prevents those approaches to shed the lights on more accurate discoveries. Application of such models on domain-specific KGs takes advantage of enriched meta-data and provides accurate results where the underlying domain can enormously benefit. In this work, the TransE embedding model is reconciled for a specific link prediction task on scholarly metadata. The results show a significant shift in the accuracy and performance evaluation of the model on a dataset with scholarly metadata. The newly proposed version of TransE obtains 99.9% for link prediction task while original TransE gets 95%. In terms of accuracy and Hit@10, TransE outperforms other embedding models such as ComplEx, TransH and TransR experimented over scholarly knowledge graphs Citation Count: 12 - Paper ID: 3a6933886485038805e7af23a38845f9d93e1b9c Title: Query Answering from Traditional Dance Videos: Case Study of Zapin Dances Abstract: The aim of this paper is to highlight two important issues related to the annotation and querying of Intangible Cultural Heritage video datasets. First, we focus on ontology completion by annotating dance videos. In order to build video training sets and to enrich the proposed ontology, manual video annotation is performed based on background knowledge formalized in an ontology, representing a semantics of a traditional dance. The paper provides a case study on Malaysian Zapin dances. Second, we address the question of how can end-users efficiently query the datasets of annotated videos that are built. Citation Count: 4 - Paper ID: c70cefca5c76724b27e4615259be889d9ca3aa92 Title: AQuA: An Adversarially Authored Question-Answer Dataset for Common Sense Abstract: Commonsense reasoning is a critical AI capability, but it is difficult to construct challenging datasets that test common sense. Recent neural question answering systems, based on large pre-trained models of language, have already achieved near-human-level performance on commonsense knowledge benchmarks. These systems do not possess human-level common sense, but are able to exploit limitations of the datasets to achieve human-level scores. We introduce the CODAH dataset, an adversarially-constructed evaluation dataset for testing common sense. CODAH forms a challenging extension to the recently-proposed SWAG dataset, which tests commonsense knowledge using sentence-completion questions that describe situations observed in video. To produce a more difficult dataset, we introduce a novel procedure for question acquisition in which workers author questions designed to target weaknesses of state-of-the-art neural question answering systems. Workers are rewarded for submissions that models fail to answer correctly both before and after fine-tuning (in cross-validation). We create 2.8k questions via this procedure and evaluate the performance of multiple state-of-the-art question answering systems on our dataset. We observe a significant gap between human performance, which is 95.3%, and the performance of the best baseline accuracy of 67.5% by the BERT-Large model. Citation Count: 1 - Paper ID: 42d83576ee920c1b6df318e212047d9ba57fc4fd Title: BAMBOO: A Comprehensive Benchmark for Evaluating Long Text Modeling Capacities of Large Language Models Abstract: Large language models (LLMs) have achieved dramatic proficiency over NLP tasks with normal length. Recently, multiple studies have committed to extending the context length and enhancing the long text modeling capabilities of LLMs. To comprehensively evaluate the long context ability of LLMs, we propose BAMBOO, a multi-task long context benchmark. BAMBOO has been designed with four principles: comprehensive capacity evaluation, avoidance of data contamination, accurate automatic evaluation, and different length levels. It consists of 10 datasets from 5 different long text understanding tasks, i.e., question answering, hallucination detection, text sorting, language modeling, and code completion, to cover various domains and core capacities of LLMs. We conduct experiments with five widely-used long-context models and further discuss five key questions for long text research. In the end, we discuss problems of current long-context models and point out future directions for enhancing long text modeling capacities. We release our data, prompts, and code at https://anonymous.4open.science/r/BAMBOO/. Citation Count: 8 - Paper ID: 7fff3df60b8d9a5e57627f4eb6b3022b60f3a39d Title: An Investigation of LLMs' Inefficacy in Understanding Converse Relations Abstract: Large Language Models (LLMs) have achieved remarkable success in many formal language oriented tasks, such as structural data-to-text and semantic parsing. However current benchmarks mostly follow the data distribution of the pre-training data of LLMs. Therefore, a natural question rises that do LLMs really understand the structured semantics of formal languages. In this paper, we investigate this problem on a special case, converse binary relation. We introduce a new benchmark ConvRe focusing on converse relations, which contains 17 relations and 1240 triples extracted from popular knowledge graph completion datasets. Our ConvRE features two tasks, Re2Text and Text2Re, which are formulated as multi-choice question answering to evaluate LLMs' ability to determine the matching between relations and associated text. For the evaluation protocol, apart from different prompting methods, we further introduce variants to the test text and few-shot example text. We conduct experiments on three popular LLM families and have observed various scaling trends. The results suggest that LLMs often resort to shortcut learning and still face challenges on our proposed benchmark. Citation Count: 4

Two is Better than Many? Binary Classification as an Effective Approach to Multi-Choice Question Answering Deepanway Ghosal DeCLaRe Navonil M... Two is Better than Many? Binary Classification as an Effective Approach to Multi-Choice Question Answering Deepanway Ghosal DeCLaRe Navonil Majumder DeCLaRe Rada Mihalcea Soujanya Poria DeCLaRe DeCLaRe DeCLaRe Lab, Singapore University of Technology and Design, Singapore University of Michigan, USA {deepanway_ghosal@mymail.,navonil_majumder@,sporia@}sutd.edu.sg mihalcea@umich.edu Abstract We propose a simple refactoring of multi- choice question answering (MCQA) tasks as a series of binary classifications. The MCQA task is generally performed by scoring each (question, answer) pair normalized over all the pairs, and then selecting the answer from the pair that yield the highest score. For nanswer choices, this is equivalent to an n-class clas- sification setup where only one class (true an- swer) is correct. We instead show that clas- sifying (question, true answer) as positive in- stances and (question, false answer) as neg- ative instances is significantly more effective across various ... Two is Better than Many? Binary Classification as an Effective Approach to Multi-Choice Question Answering Deepanway Ghosal DeCLaRe Navonil Majumder DeCLaRe Rada Mihalcea Soujanya Poria DeCLaRe DeCLaRe DeCLaRe Lab, Singapore University of Technology and Design, Singapore University of Michigan, USA {deepanway_ghosal@mymail.,navonil_majumder@,sporia@}sutd.edu.sg mihalcea@umich.edu Abstract We propose a simple refactoring of multi- choice question answering (MCQA) tasks as a series of binary classifications. The MCQA task is generally performed by scoring each (question, answer) pair normalized over all the pairs, and then selecting the answer from the pair that yield the highest score. For nanswer choices, this is equivalent to an n-class clas- sification setup where only one class (true an- swer) is correct. We instead show that clas- sifying (question, true answer) as positive in- stances and (question, false answer) as neg- ative instances is significantly more effective across various models and datasets. We show the efficacy of our proposed approach in differ- ent tasks – abductive reasoning, commonsense question answering, science question answer- ing, and sentence completion. Our DeBERTa binary classification model reaches the top or close to the top performance on public leader- boards for these tasks. The source code of the proposed approach is available at https: //github.com/declare-lab/TEAM . 1 Introduction Starting with the early Text Retrieval Conference (TREC) community-wide evaluations of textual question answering (V oorhees et al., 1999), all the way to the recent work on multimodal question answering (Lei et al., 2018; Tapaswi et al., 2016; Jang et al., 2017; Castro et al., 2020) and common- sense question answering (Sap et al., 2019; Talmor et al., 2019), the task has become a staple of the natural language processing research community. One of the major challenges encountered in ques- tion answering is the evaluation, which often re- quires human input to evaluate the textual answers thoroughly. Because of this, the alternative that has been proposed is that of multi-choice question answering , where the correct answer is provided together with other incorrect answers. The task is thus transformed into that of answer classifica- tion, where a system has to select one answer fromthe choices provided. While there are drawbacks associated with this evaluation metric, it has been widely adopted because of its benefit of providing a clear evaluation methodology. In this paper, we reformulate the task of multi-choice question answering as a binary classification task and show that this re-framing leads to significant performance improvements on several datasets. Importantly, this formulation brings flexibility to the overall question-answering setup, as it reduces the dependence on the up-front availability of multiple candidate answers. Using our method – TEAM (Two is b Etter th AnMany), candidate answers can be produced and evaluated for correctness on the fly, and thus the answer classification component can be also used in conjunction with more natural settings that use open-ended answer generation (Castro et al., 2022; Sadhu et al., 2021). 2 Methodology Letqbe a question for which multiple answer choicesA=fa1; : : : ; a ngare given. Optionally, there is some context cwhich could be helpful for answering the question. The objective is to select the correct answer akfrom the answer set A. For some of the datasets used in the paper, the question qis not provided, and the answer is based only on the context c. For example, SWAG and HellaSwag are two such datasets where the task is to choose the best possible ending for sentence completion, as shown in Table 1. In this case, the question qcan be assumed as implicit: What is the best possible ending for the context? The sentence to be completed is considered as the context c. We discuss how the MCQA task is generally performed using transformer language models in §2.1. We denote this approach as Score-based Method orScore method . We then discuss our proposed Binary Classification-based Method , TEAM in §2.2.arXiv:2210.16495v1 [cs.CL] 29 Oct 20222.1 Score-based Method ( Score ) We use the notation introduced earlier in §2. Given question q, optional context c, and the an- swer choicesA=fa1; a2; : : : ; a ng,ndifferent in- put sequences are constructed each containing the concatenation of the question q, context c, and one possible answer choice ai. The sequences are in- dependently encoded through a pre-trained trans- former language model such as RoBERTa (Liu et al., 2019) or DeBERTa (He et al., 2021). A score siis predicted for each input sequence which is then normalized with a softmax layer across the n outputs to obtain score qi. The cross-entropy loss is used to train the en- coder model. Assuming the answer akis correct, the loss can be obtained as follows: L=nX i=1pilog(qi) =log(qk) (1) where piare considered as the class labels. The class pkcorresponding to the gold answer akis valued as 1, and all other classes are valued as 0. The loss is equivalent to the cross-entropy loss in an-class classification setup. The normalization of the scores using the softmax layer to obtain a distribution over the answer choices is also analo- gous to the probability distribution over the differ- ent classes in the multi-class classification setup. The choice providing the highest score is the predicted answer during inference. The Score method was used for the SWAG task in BERT (De- vlin et al., 2019), StoryCloze task in GPT (Radford et al., 2018) and has been used for all MCQA tasks in the huggingface transformers1framework. 2.2 Classification-based Method ( TEAM ) For our proposed classification-based method, we first extend the pre-trained language model by adding a classification head with two nodes. The values of these two nodes will denote the un- normalized scores for the negative and positive classes in our classification setup. Now, similar to the previous Score method, we first construct ndifferent input sequences by concatenating the question q, the optional con- textc, and each possible answer choice ai. We then obtain the unnormalized negative and posi- tive scores s iands+ ifor each sequence by in- dependently encoding them through the modified language model. We normalize each pair of scores 1https://github.com/huggingface/ transformersthrough a softmax layer to obtain probabilities of negative and positive classes: q iandq+ i, respec- tively. We consider the sequence corresponding to the gold answer akas positive, and all the other se- quences as negative. Therefore, the loss function takes the following form: L=nX i=1(p+ ilog(q+ i) +p ilog(q i)) =log(q+ k)nX i=1;i6=klog(q i)(2) where p+ iandp iare considered as the class la- bels. As akis the gold answer, we use p+ k= 1, p k= 0andp+ i= 0,p i= 1, when i6=k. Although Eq. (2) is a suitable loss function for single correct answer cases, it can be easily ex- tended for instances or datasets with multiple cor- rect answers. This can be done by changing the class labels p+ iandp ito positive and negative ap- propriately for the additional correct answers. During inference, we choose the answer with the highest positive class probability as the pre- dicted answer. We will show later in §4 that the TEAM method generally outperforms the Score method across several datasets for the same choice of transformer models. 3 Experimental Datasets We experiment with the following datasets: Abductive NLI (Bhagavatula et al., 2020). Given two observations o1ando2(considered as con- textc), the goal is to select the more plausible intermediate event among hypotheses h1andh2. We use the sequences fo1; h1; o2gandfo1; h2; o2g as input for both the Score andTEAM method. Assuming h1is the gold answer, we classify fo1; h1; o2gas positive;fo1; h2; o2gas negative. CommonsenseQA (Talmor et al., 2019) or CQA is a dataset for commonsense QA based on knowl- edge encoded in ConceptNet (Speer et al., 2017). Given a question, there are five possible choices , among which only one is correct. We do not use any additional knowledge or context for this task. CommonsenseQA 2.0 (Talmor et al., 2021) or CQA2 is a recent challenging QA dataset collected with a model-in-the-loop approach. The dataset contains commonsense questions from various reasoning categories with either yesornoanswer. QASC (Khot et al., 2020) or Question Answer-Dataset Instance CQAQuestion: Where on a river can you hold a cup upright to catch water on a sunny day? Choice 1: Waterfall Choice 2: Bridge : : : Choice 5: Mountain QASCQuestion: Differential heating of air can be harnessed for what? Choice 1: electricity production Choice 2: running and lifting Choice 3: animal survival : : : Choice 8: reducing acid rain SWAGPartial Event: On stage, a woman takes a seat at the piano. She Ending 1: sits on a bench as her sister plays with the doll. : : : Ending 4: nervously sets her fingers on the keys. PIQAGoal: To separate egg whites from the yolk using a water bottle, you should Solution 1: Squeeze the water bottle and press it against the yolk. Release, which creates suction and lifts the yolk. Solution 2: Place the water bottle and press it against the yolk. Keep pushing, which creates suction and lifts the yolk. Table 1: Illustration of some of the datasets used in this work. The answers highlighted in green are the correct an- swers. CQA: Commonsense QA, PIQA: Physical IQA. ing via Sentence Composition task requires fact retrieval from a large corpus and composing them to answer a multi-choice science question. Each question qhas eight choices, among which one is correct. We use the question and choices with- out any retrieved facts for this task. We evalu- ate another task setup QASC-IR (information re- trieval) where we use two-step IR retrieved facts as in Khot et al. (2020) as additional context c. SWAG, HellaSwag (Zellers et al., 2018, 2019) are two datasets for grounded commonsense in- ference, where the objective is to find the correct ending given a partial description of an event. We consider the partial description as the context c. The correct ending is to be chosen from a pool of four possible choices. Social IQA (SIQA) (Sap et al., 2019) is a dataset for commonsense reasoning about social interac- tive situations. Given a question about a social sit- uation context, the objective is to select the correct answer from three possible choices. Physical IQA (PIQA) (Bisk et al., 2020) is de- signed to investigate physical knowledge of lan- guage models. The task is to select the correct so- lution for a goal from two given choices. CosmosQA (Huang et al., 2019) is a QA dataset for commonsense-based reading comprehension. Given a question about a paragraph ( c), the task is to select the correct answer among four choices. CICERO v1, v2 (Ghosal et al., 2022; Shen et al., 2022) are datasets for contextual commonsense reasoning in dialogues. Given the dialogue and a question about an utterance, the task is to choose the correct answer among multiple choices. We modify the original datasets to use them in aMCQA setup. More details are in the appendix. 4 Results We use the RoBERTa Large (Liu et al., 2019) and DeBERTa Large (He et al., 2021) model to benchmark the Score andTEAM method across the experimental datasets. We report the accuracy for the validation set in Table 2 and accuracy of leaderboard submissions for the test set in Table 3. We also report results for other QA systems such as UnifiedQA (Khashabi et al., 2020) and UNI- CORN (Lourie et al., 2021) for the test set (wher- ever available) in Table 3. Our main finding is that the TEAM method im- proves over the Score method for most of the datasets except Social IQA, Physical IQA, and CI- CERO v1. We observe this result for both the RoBERTa and DeBERTa models. Abductive Reasoning: The improvement is con- sistently large for both validation and test set in the Abductive NLI (ANLI) dataset. The problem of intermediate hypothesis selection transforms into a problem of plausible story selection as we use the sequencefo1; h; o 2gas our input. In this for- mulation, the TEAM method is significantly better than the Score method for both RoBERTa and DeBERTa models. Science QA: We also observe considerable im- provements in the QASC dataset without and with the additional retrieved knowledge. The RoBERTa- TEAM model is more than 7% better in the test set when retrieved knowledge is not used. The difference in performance is around 3% and 4.5% in the validation and test set when the re- trieved knowledge is used. For DeBERTa, we ob- serve the most significant improvement in the test results of the QASC-IR setting, where the TEAM method is 3.7% better than the Score method. Commonsense QA and Sentence Ending Pre- diction: TheTEAM method is also better than theScore method for commonsense question- answering in CommonsenseQA and Common- senseQA 2.0 across most settings. One notable instance is the 3% superior score of the De- BERTa TEAM in the CommonsenseQA 2.0 vali- dation set. We observe a similar trend in results for sentence-ending prediction in SWAG and Hel- laSwag. The improvement in performance for the TEAM method is between 0.85-1.9% in the test set. We also notice improvements in the test set results for reading comprehension QA in CosmosQA.Model Method ANLI CQA CQA2 QASC QASC-IR SWAG H-SWAG SIQA PIQA CosmosQACICERO v1 v2 RoBERTa LargeScore 85.25 73.63 54.76 53.46 77.21 89.23 83.89 78.15 78.89 80.44 80.33 85.25 TEAM 87.47 75.32 55.83 57.24 80.35 89.49 84.52 76.49 76.71 80.37 77.54 86.53 DeBERTa LargeScore 89.75 83.75 66.63 74.41 89.31 93.14 94.67 80.82 87.81 86.13 86.60 89.06 TEAM 92.23 83.34 69.57 75.33 91.09 93.27 95.47 80.27 86.07 86.35 84.48 90.59 Table 2: Accuracy on the validation split of the datasets. All numbers are the average of five runs with different seeds. Model Method ANLI CQA2 QASC QASC-IR SWAG H-SWAG SIQA PIQA CosmosQACICERO v1 v2 RoBERTa LargeScore 83.91 55.44 46.52 73.26 88.97 81.70 76.70 79.40 80.71 83.28 89.61 TEAM 87.04 56.73 53.80 77.93 89.88 (7) 83.63 75.96 74.55 80.84 79.94 89.81 DeBERTa LargeScore 89.74 67.37 71.74 85.65 92.37 (2) 94.72 (4) 80.18 87.41 (4) 85.51 88.04 92.67 TEAM 92.20 (1) 68.38 (9) 74.35 89.35 (3) 94.12 (1) 95.57 (2) 79.89 85.90 (5) 86.86 (5) 86.84 93.25 UnifiedQA 11B - - - 78.50 89.60 - - 81.40 89.50 - - - UNICORN 11B - 87.30 70.20 - - - 93.90 83.20 90.10 91.80 - - Table 3: Accuracy on the test split of the datasets. Numbers on the parentheses indicate rank on the leaderboard (if in the top 10) at the time of submission to the leaderboard. Numbers in purple indicate results for RoBERTa Large as reported in the UNICORN paper (Lourie et al., 2021). We do not report results for CommonsenseQA (CQA) test set as test labels are not publicly available and there is no automated submission leaderboard. Dialogue Commonsense Reasoning: We observe contrasting results in CICERO v1 and v2. The Score method outperforms the TEAM method by around 2-3% in CICERO v1. However, the TEAM method is better in CICERO v2 for both RoBERTa and DeBERTa models. We analyze the results in more detail in §5.1. Negative Results: TheScore method outper- forms the TEAM method in Physical IQA (PIQA) and CICERO v1. These two datasets contain an- swer choices that are lexically close together and subtly different from each other (example in Ta- ble 1). We analyze the results in more detail in §5.1. The Score method is also the better per- forming method in SIQA, with small improve- ments over the TEAM method in DeBERTa and comparatively large improvements in RoBERTa. We surmise that the Score method is better be- cause the dataset contains complex social com- monsense scenarios, for which learning by di- rectly comparing the options is more effective. State-of-the-Art Models and Leaderboard Sub- missions: We also report the results for Uni- fiedQA and UNICORN 11B models for the test set in Table 3. We compare these results against our best-performing model: DeBERTa Large in classification setup (DeBERTa- TEAM ). DeBERTa- TEAM maintains parity with UnifiedQA 11B in QASC-IR, despite being 36 times smaller. UNI- CORN 11B outperforms DeBERTa- TEAM by a large margin on SIQA, PIQA, and CosmosQA.It is an expected result as UNICORN is trained on multiple datasets for commonsense reasoning starting from the T5-11B checkpoint and then fine- tuned on each target dataset. DeBERTa- TEAM is, however, considerably better in Abductive NLI and HellaSwag. DeBERTa- TEAM also reached the top or close to the top of the leaderboard (at the time of submission to the leaderboard) in Abduc- tive NLI, SWAG, HellaSwag, and QASC. 5 Analysis 5.1 How Does Similar Answer Choices Affect Performance? We analyze the similarity between the correct and incorrect choices to understand why the TEAM method is better than the Score method in most of the datasets and vice-versa in the others. We report the lexical similarity with BLEU (Papineni et al., 2002), ROUGE-L (Lin, 2004), and semantic similarity with all-mpnet-base-v2 sentence trans- former (Reimers and Gurevych, 2019) in Table 4. We also report the difference in performance be- tween TEAM andScore models for RoBERTa and DeBERTa in the
columns. The similarity measurements in Table 4 indicate that the datasets can be clearly segregated into two groups – one with low to medium similarity, and the other with very high similarity. Interestingly, the
values are mostly positive for the low to medium similarity group, and all negatives for the high similarity group. We surmise that the differ- ence between the very similar correct and incor-Dataset BLEU1 BLEU4 ROUGE Sem-Sim
1
2 ANLI 21.84 7.81 24.61 46.02 2.22 2.48 CQA 1.48 0 1.31 30.75 1.69 -0.41 QASC 3.15 0.95 2.08 25.71 3.14 1.78 SWAG 12.78 0.81 11.61 30.47 0.26 0.13 H-SWAG 18.55 1.18 16.14 46.95 0.63 0.80 SIQA 12.56 3.99 10.41 29.17 -1.66 -0.55 CosmosQA 32.37 13.31 24.66 35.29 -0.07 0.22 CICEROv2 30.00 7.50 33.85 44.23 1.28 1.53 PIQA 81.97 72.77 74.01 82.50 -2.18 -1.74 CICEROv1 73.17 53.96 74.98 74.12 -2.79 -2.12 Table 4: Average similarity between correct and incorrect answer choices in the validation set for different datasets. Numbers are shown on a scale of 0-100.
1and
2indi- cate difference in performance between TEAM andScore methods for RoBERTa and DeBERTa in validation set. rect choices are better captured through the soft- max activation over the answers in the Score method. However, this aspect is not captured in theTEAM method, as sequences corresponding to the correct and incorrect choices are separately classified as positive or negative. Thus, the Score method is more effective when the answer choices are very similar, as in PIQA or CICERO v1. 5.2 How Accurate is the Binary Classifier? We evaluate how often input sequences corre- sponding to correct and incorrect answers are pre- dicted accurately with DeBERTA- TEAM binary classification model in Table 5. The binary classi- fier model is more likely to predict all answers as negative than all answers as positive, as it learns from more negative choices in most datasets. In- terestingly, however, the model predicts all posi- tive answers for 25.63% instances in PIQA, which is significantly higher than all the other datasets. This is one of the sources of error in PIQA, as the model often predicts both choices as positive, but assigns a higher positive probability to the incor- rect choice. We also report the % of instances for which the correct answer is predicted as positive and all incorrect answers are predicted as negative in the Accurate column. The accuracy is high- est in HellaSWAG and lowest in QASC, which co- relates well with the highest performance in Hel- laSWAG and second lowest performance in QASC across the datasets in Table 2 and Table 3. 5.3 Error Analysis We show some examples of incorrect predictions for the DeBERTa- TEAM model in the Common- senseQA and PIQA dataset in Table 6. The er- roneously predicted answers in CommonsenseQA are often very close in meaning to the correct an-DatasetDeBERTa- TEAM Predicted All Neg Pos Incor as Neg Cor as Pos Accurate CQA 17.69 0.08 70.35 76.99 52.66 CQA2 1.81 6.53 65.17 69.89 63.36 QASC 37.37 0.0 80.45 55.29 43.09 SWAG 13.2 0.05 86.97 85.0 73.77 H-SWAG 15.63 0.01 94.69 83.39 79.06 SIQA 20.93 2.61 73.69 72.36 52.76 PIQA 19.37 25.63 70.46 76.71 51.09 CosmosQA 19.33 0.2 78.32 76.21 58.99 CICEROv1 22.62 0.37 80.60 71.80 57.44 CICEROv2 11.26 2.64 79.40 85.71 68.14 Table 5: DeBERTA- TEAM binary classification results. The Neg andPoscolumn indicate % of instances for which all answer choices are predicted as negative or positive. The In- cor as Neg ,Cor as Pos , and Accurate column indicate % of instances for which all incorrect answers are predicted as negative, the correct answer is predicted as positive, and all answers are predicted accurately as negative or positive. Ac- curate is the intersection of Incor as Neg andCor as Pos . swers. Furthermore, the incorrectly predicted an- swer could also be argued as correct for some in- stances (second example in Table 6), as the incor- rect choice is also equally plausible. In PIQA how- ever, the model make mistakes where complex sci- entific and physical world knowledge is required. The incorporation of external knowledge is likely necessary to answer these questions accurately. Dataset : CommonsenseQA. Question : Though the thin film seemed fragile, for it’s intended purpose it was actually nearly what? Correct Answer : Indestructible. Predicted Answer : Unbreakable. Dataset : CommonsenseQA. Question : She was always helping at the senior center, it brought her what? Correct Answer : Happiness. Pre- dicted Answer : Satisfaction. Dataset : PIQA. Goal : To discourage house flies from living in your home, Correct Answer : keep basil plants in the kitchen or windows. Predicted Answer : keep lavender plants in the kitchen or window. Dataset : PIQA. Goal : To cook perfectly golden pancakes, Correct An- swer : keep the temperature low for a longer time. Predicted Answer : keep the temperature high and cook quickly. Table 6: Some examples of incorrect predictions in Com- monsenseQA and PIQA. 6 Conclusion In this paper, we introduced a simple binary classification method as an alternative way to ad- dress multi-choice question answering (MCQA) tasks. Through evaluations on ten different MCQA benchmarks, we showed that this simple method generally exceeds the performance of the score-based method traditionally used in the past. We believe this approach can also be used in the more natural open-ended answer generation setups, thus providing a “bridge” between the MCQA and answer generation frameworks for question answering.7 Limitations Although the method we introduced is more flex- ible than the answer scoring approach typically used for MCQA, it still lacks the full flexibility of open-ended question answering and assumes the availability of a candidate answer that it can clas- sify as correct or incorrect. Additionally, even if our approach outperforms the score-based methods for most of the bench- marks we considered, there are still some datasets (e.g., SIQA, PIQA, CICERO v1), where the score- based method performs best. We leave it for fu- ture work to identify a principled approach for se- lecting the best methodology to use for a given dataset. Acknowledgement This research/project is supported by the National Research Foundation, Singapore, and the Min- istry of National Development, Singapore under its Cities of Tomorrow R&D Programme (CoT Award COT-V2-2020-1). Any opinions, find- ings, and conclusions, or recommendations ex- pressed in this material are those of the author(s) and do not reflect the views of the National Re- search Foundation, Singapore, and the Ministry of National Development, Singapore. This re- search is also supported by A*STAR under its RIE 2020 AME programmatic grant RGAST2003 and the Ministry of Education, Singapore, under its AcRF Tier-2 grant (Project no. T2MOE2008, and Grantor reference no. MOET2EP20220-0017). Any opinions, findings, conclusions, or recom- mendations expressed in this material are those of the author(s) and do not reflect the views of the Ministry of Education, Singapore. References Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Han- nah Rashkin, Doug Downey, Wen-tau Yih, and Yejin Choi. 2020. Abductive commonsense reasoning. In ICLR . Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. 2020. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence , vol- ume 34, pages 7432–7439. Santiago Castro, Mahmoud Azab, Jonathan Stroud, Cristina Noujaim, Ruoyao Wang, Jia Deng, and Rada Mihalcea. 2020. LifeQA: A real-life datasetfor video question answering. In Proceedings of the 12th Language Resources and Evaluation Con- ference , pages 4352–4358, Marseille, France. Euro- pean Language Resources Association. Santiago Castro, Ruoyao Wang, Pingxuan Huang, Ian Stewart, Oana Ignat, Nan Liu, Jonathan Stroud, and Rada Mihalcea. 2022. FIBER: Fill-in-the-blanks as a challenging video understanding evaluation frame- work. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers) , pages 2925–2940, Dublin, Ireland. Association for Computational Linguistics. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pages 4171–4186, Minneapolis, Minnesota. Associ- ation for Computational Linguistics. Deepanway Ghosal, Siqi Shen, Navonil Majumder, Rada Mihalcea, and Soujanya Poria. 2022. Cicero: A dataset for contextualized commonsense infer- ence in dialogues. In Proceedings of the 60th An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 5010– 5028. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2021. Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing. arXiv preprint arXiv:2111.09543 . Lifu Huang, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2019. Cosmos qa: Machine reading comprehension with contextual commonsense rea- soning. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natu- ral Language Processing (EMNLP-IJCNLP) , pages 2391–2401. Yunseok Jang, Yale Song, Youngjae Yu, Youngjin Kim, and Gunhee Kim. 2017. TGIF-QA: Toward spatio- temporal reasoning in visual question answering. InProceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition (CVPR) , pages 2758–2766. Daniel Khashabi, Sewon Min, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter Clark, and Han- naneh Hajishirzi. 2020. UNIFIEDQA: Crossing for- mat boundaries with a single QA system. In Find- ings of the Association for Computational Linguis- tics: EMNLP 2020 , pages 1896–1907, Online. As- sociation for Computational Linguistics. Tushar Khot, Peter Clark, Michal Guerquin, Peter Jansen, and Ashish Sabharwal. 2020. Qasc: A dataset for question answering via sentence compo- sition. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 34, pages 8082–8090.Jie Lei, Licheng Yu, Mohit Bansal, and Tamara Berg. 2018. TVQA: Localized, compositional video ques- tion answering. In Proceedings of the 2018 Con- ference on Empirical Methods in Natural Language Processing , pages 1369–1379, Brussels, Belgium. Association for Computational Linguistics. Chin-Yew Lin. 2004. Rouge: A package for auto- matic evaluation of summaries. In Text summariza- tion branches out , pages 74–81. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprint arXiv:1907.11692 . Ilya Loshchilov and Frank Hutter. 2018. Decoupled weight decay regularization. In International Con- ference on Learning Representations . Nicholas Lourie, Ronan Le Bras, Chandra Bhagavat- ula, and Yejin Choi. 2021. Unicorn on rainbow: A universal commonsense reasoning model on a new multitask benchmark. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 13480–13488. Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: a method for automatic eval- uation of machine translation. In Proceedings of the 40th annual meeting of the Association for Compu- tational Linguistics , pages 311–318. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language under- standing by generative pre-training. Nils Reimers and Iryna Gurevych. 2019. Sentence- bert: Sentence embeddings using siamese bert- networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Process- ing. Association for Computational Linguistics. Arka Sadhu, Kan Chen, and Ram Nevatia. 2021. Video question answering with phrases via semantic roles. InProceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies , pages 2460–2478, Online. Association for Compu- tational Linguistics. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. 2019. Social iqa: Com- monsense reasoning about social interactions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan- guage Processing (EMNLP-IJCNLP) , pages 4463– 4473. Siqi Shen, Deepanway Ghosal, Navonil Majumder, Henry Lim, Rada Mihalcea, and Soujanya Poria. 2022. Multiview contextual commonsense infer- ence: A new dataset and task. arXiv preprint arXiv:2210.02890 .Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5: An open multilingual graph of general knowledge. In Thirty-first AAAI confer- ence on artificial intelligence . Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. Commonsenseqa: A ques- tion answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pages 4149–4158. Alon Talmor, Ori Yoran, Ronan Le Bras, Chandra Bha- gavatula, Yoav Goldberg, Yejin Choi, and Jonathan Berant. 2021. Commonsenseqa 2.0: Exposing the limits of ai through gamification. In Thirty-fifth Conference on Neural Information Processing Sys- tems Datasets and Benchmarks Track (Round 1) . Makarand Tapaswi, Yukun Zhu, Rainer Stiefelhagen, Antonio Torralba, Raquel Urtasun, and Sanja Fidler. 2016. MovieQA: Understanding stories in movies through question-answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 4631–4640. Ellen M V oorhees et al. 1999. The trec-8 question an- swering track report. In Trec, volume 99, pages 77– 82. Rowan Zellers, Yonatan Bisk, Roy Schwartz, and Yejin Choi. 2018. SWAG: A large-scale adversar- ial dataset for grounded commonsense inference. InProceedings of the 2018 Conference on Empiri- cal Methods in Natural Language Processing , pages 93–104, Brussels, Belgium. Association for Compu- tational Linguistics. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? In Proceed- ings of the 57th Annual Meeting of the Association for Computational Linguistics , pages 4791–4800.A Experimental Details We train all the score-based and classification- based models with the AdamW (Loshchilov and Hutter, 2018) optimizer with a learning rate of 1e- 6, 3e-6, 5e-6, 1e-5, 3e-5. We train all the models for 8 epochs. The best models are chosen based on the results on the validation set. The RoBERTa- Large and DeBERTa-Large models have 355M and 304M parameters, respectively. B Computational Resources We use a single Quadro RTX 8000 GPU for our experiments. Training takes between 30 minutes to 8 hours for the different datasets used in the pa- per. C Dataset Details All datasets used in this paper are in English lan- guage. The datasets are available in the cor- responding leaderboard websites2or through the huggingface datasets hub3. The number of MCQA instances in the training, validation and test set of the various datasets are shown in Table 7. Some example instances from the datasets are shown in Table 8. Dataset Train Validation Test Abductive NLI 169,654 1,532 3,040 Commonsense QA 9,741 1,221 1,140 Commonsense QA 2.0 9,264 2,541 2,473 QASC / QASC IR 8,134 926 920 SWAG 73,546 20,006 20,005 HellaSwag 39,905 10,042 10,050 PIQA 16,113 1,838 3,446 SIQA 33,410 1,954 2,059 CosmosQA 25,262 2,985 6,963 CICERO v1 27,225 9,470 9,064 CICERO v2 13,496 2,806 4,150 Table 7: Number of MCQA instances in the train, vali- dation, and test set for the experimental datasets. D Modifications in CICERO CICERO v1 and v2 both contain instances with ei- ther one or more than one correct answer choices. We make the following modifications in the origi- nal datasets to use them in our MCQA setup here, as we assume only one answer is correct for a given MCQA instance: 2https://leaderboard.allenai.org/ 3https://huggingface.co/datasetsv1:We only consider instances which has one an- notated correct answer. Each instance in CICERO v1 has five possible answer choices. Thus, the instances selected for our experiments in all the three sets (training, validation, and test split) has one correct answer and four incorrect answers. v2:All instances in CICERO v2 has at-least two correct answers. We consider instances with at- least one incorrect answer and create the MCQA dataset as follows: • If the original CICERO v2 instance has ncor- rect answers, then we will create nMCQA instances from it, each having one of the cor- rect answers and three incorrect answers. • The three incorrect answers will be chosen from the incorrect answers of the original in- stance. We perform oversampling (some in- correct answers repeated) to create three in- correct answers if there are less than three in- correct answers in the original instance. For example, an instance in CICERO v2 has an- swer choices:fc1; c2; i1; i2g. The correct answers arefc1; c2gand the incorrect answers are fi1; i2g. We create two MCQA instances from the original instance – i) with answer choices fc1; i1; i2; i1g, and ii) with answer choices fc2; i1; i2; i2g.Dataset Task Instance ANLI Intermediate Event SelectionEvent 1: Jenny cleaned her house and went to work, leaving the window just a crack open. Event 2: When Jenny returned home she saw that her house was a mess! Choice 1: A thief broke into the house by pulling open the window. Choice 2: At work, she opened her window and the wind blew her papers everywhere. CommonsenseQA Answer SelectionQuestion: Where on a river can you hold a cup upright to catch water on a sunny day? Choice 1: Waterfall Choice 2: Bridge Choice 3: Valley Choice 4: Pebble Choice 5: Mountain CommonsenseQA 2.0 Answer SelectionQuestion: The peak of a mountain almost always reaches above the the tree line. Choice 1: No Choice 2: Yes QASC Answer SelectionQuestion: Differential heating of air can be harnessed for what? Choice 1: electricity production Choice 2: running and lifting Choice 3: animal survival : : : Choice 8: reducing acid rain SWAG Ending PredictionPartial Event: On stage, a woman takes a seat at the piano. She Ending 1: sits on a bench as her sister plays with the doll. Ending 2: smiles with someone as the music plays. Ending 3: is in the crowd, watching the dancers. Ending 4: nervously sets her fingers on the keys. HellaSwag Ending PredictionPartial Event: A woman is outside with a bucket and a dog. The dog is running around trying to avoid a bath. She Ending 1: rinses the bucket off with soap and blow dry the dog’s head. Ending 2: uses a hose to keep it from getting soapy. Ending 3: gets the dog wet, then it runs away again. Ending 4: gets into a bath tub with the dog. Social IQA Answer SelectionContext: Alex spilled the food she just prepared all over the floor and it made a huge mess. Question: What will Alex want to do next? Choice 1: taste the food Choice 2: mop up Choice 3: run around in the mess Physical IQA Solution SelectionGoal: To separate egg whites from the yolk using a water bottle, you should Solution 1: Squeeze the water bottle and press it against the yolk. Release, which creates suction and lifts the yolk. Solution 2: Place the water bottle and press it against the yolk. Keep pushing, which creates suction and lifts the yolk. CosmosQA Answer SelectionContext: : It’s a very humbling experience when you need someone to dress you every morning, tie your shoes, and put your hair up. Every menial task takes an unprecedented amount of effort. It made me appreciate Dan even more. But anyway I shan’t dwell on this (I’m not dying after all) and not let it detract from my lovely 5 days with my friends visiting from Jersey Question: What’s a possible reason the writer needed someone to dress him every morning? Chocie 1: The writer doesn’t like putting effort into these tasks. Chocie 2: The writer has a physical disability. Chocie 3: The writer is bad at doing his own hair. Chocie 4: None of the above choices. CICERO v2 Answer SelectionDialogue: A: Dad, why are you taping the windows? B: Honey, a typhoon is coming. A: Really? Wow, I don’t have to go to school tomorrow. B: Jenny, come and help, we need to prepare more food. A: OK. Dad! I’m coming. Target: Jenny, come and help, we need to prepare more food. Question: What subsequent event happens or could happen following the target? Chocie 1: Jenny and her father stockpile food for the coming days. Chocie 2: Jenny and her father give away all their food. Chocie 3: Jenny and her father eat all the food in their refrigerator. Chocie 4: Jenny and her father eat all the food in their refrigerator. Table 8: Illustration of the different datasets used in this work. The answers highlighted in green are the correct answers.

Paper 9342e654bcb015240bfa934587a830ab6bffa97e not found in buffer: ['410e4e70ec38f9a4cfd43df6dd92adc9f739ce2a', '40e3cd767c9... Paper 9342e654bcb015240bfa934587a830ab6bffa97e not found in buffer: ['410e4e70ec38f9a4cfd43df6dd92adc9f739ce2a', '40e3cd767c9520171ac8ecbd2e27f3a1bf043b61', 'd777d15677b673fe02750333181ea5e1e6dadf17', 'fe351bc1bda19583a7d6b1a96a9b18f59c40b213', '441068ef17c3332d4f0af6c146e37bd8748bb90e', '781402fae3a7a44b90855eeba8e96a872347fd4b', '3a6933886485038805e7af23a38845f9d93e1b9c', 'c70cefca5c76724b27e4615259be889d9ca3aa92', '42d83576ee920c1b6df318e212047d9ba57fc4fd', '7fff3df60b8d9a5e57627f4eb6b3022b60f3a39d']. Please try a different paper. Paper 9342e654bcb015240bfa934587a830ab6bffa97e not found in buffer: ['410e4e70ec38f9a4cfd43df6dd92adc9f739ce2a', '40e3cd767c9520171ac8ecbd2e27f3a1bf043b61', 'd777d15677b673fe02750333181ea5e1e6dadf17', 'fe351bc1bda19583a7d6b1a96a9b18f59c40b213', '441068ef17c3332d4f0af6c146e37bd8748bb90e', '781402fae3a7a44b90855eeba8e96a872347fd4b', '3a6933886485038805e7af23a38845f9d93e1b9c', 'c70cefca5c76724b27e4615259be889d9ca3aa92', '42d83576ee920c1b6df318e212047d9ba57fc4fd', '7fff3df60b8d9a5e57627f4eb6b3022b60f3a39d']. Please try a different paper.

There was an error reading the PDF. Please try a different paper. There was an error reading the PDF. Please try a different paper. There was an error reading the PDF. Please try a different paper.

ScreenQA: Large-Scale Question-Answer Pairs Over Mobile App Screenshots Yu-Chung Hsiao∗†, Fedir Zubach∗, Maria Wang, Jindong Chen Google Res... ScreenQA: Large-Scale Question-Answer Pairs Over Mobile App Screenshots Yu-Chung Hsiao∗†, Fedir Zubach∗, Maria Wang, Jindong Chen Google Research yuchsiao@gmail.com ,{fedir, mariawang, jdchen}@google.com Abstract We present a new task and dataset, ScreenQA, for screen content understanding via question answering. The existing screen datasets are fo- cused either on structure and component-level understanding, or on a much higher-level com- posite task such as navigation and task com- pletion. We attempt to bridge the gap between these two by annotating 86K question-answer pairs over the RICO dataset in hope to bench- mark the screen reading comprehension capac- ity. 1 Introduction Mobile app screenshots have been analyzed using machine learning from multiple aspects. These analyses range from pixel level understanding, e.g., layout structural analyses, UI issue detection and correction (Li et al., 2022), to UI element semantics, e.g., icon recognition, button action prediction (Sunkara... ScreenQA: Large-Scale Question-Answer Pairs Over Mobile App Screenshots Yu-Chung Hsiao∗†, Fedir Zubach∗, Maria Wang, Jindong Chen Google Research yuchsiao@gmail.com ,{fedir, mariawang, jdchen}@google.com Abstract We present a new task and dataset, ScreenQA, for screen content understanding via question answering. The existing screen datasets are fo- cused either on structure and component-level understanding, or on a much higher-level com- posite task such as navigation and task com- pletion. We attempt to bridge the gap between these two by annotating 86K question-answer pairs over the RICO dataset in hope to bench- mark the screen reading comprehension capac- ity. 1 Introduction Mobile app screenshots have been analyzed using machine learning from multiple aspects. These analyses range from pixel level understanding, e.g., layout structural analyses, UI issue detection and correction (Li et al., 2022), to UI element semantics, e.g., icon recognition, button action prediction (Sunkara et al., 2022), to even higher- level functional analyses such as accessibility sup- port (Li et al., 2020c), screen description (Wang et al., 2021), and screen type classification (Deka et al., 2017). Comparatively, the content under- standing aspect is relatively understudied. By con- tent, we mean the information displayed on the screen to convey and satisfy the purpose of using the app. Examples include star ratings from restau- rant reviews, messages from chat apps, cuisine in- gredients from recipe apps, flight status and in- flight amenities from travel planner apps, etc. Hav- ing this capacity of understanding is important for two reasons: First, the sole reason for many apps and app categories to exist is to satisfy users’ in- formation need, e.g., weather, map navigation, and news apps. Second, for task completion1, which requires the eyes-free agent capacity, the two types *These authors contributed equally to this work. †The work was done when the author was with Google. 1Also referred to as automation or app control.of screen understandings — content and action un- derstanding — are inseparable in order to carry out a task successfully. Without knowing a screen state properly, a machine learning agent is unable to self-assess if the action is performed as expected, or unable to provide sufficient feedback to the user to achieve true eyes-free user experience. More in- trinsically, from a pure research perspective, we are interested in knowing the limit of machine screen content understanding2and what constitutes the challenges, given that app screenshots are entirely human artifacts made for convenient comprehen- sion. Accordingly, we annotated the RICO dataset (Deka et al., 2017) with 86,025 question- answer pairs, referred to as Screen Question Answering, or, in short, ScreenQA annotations later in this work, and released the dataset in the public domain3. The ScreenQA task requires an agent to answer a user’s question by selecting one or multiple UI elements from the given screenshot, as will be formulated in Section 2. Question answering is employed as a touchstone to sparsely4verify the quality of screen content understanding. To the best of our knowledge, this is the first large-scale questions answering dataset over mobile app screenshots, and the first one to be publicly available. Much inspired by the SQuAD dataset (Rajpurkar et al., 2016), we hope, by releasing this set of annotations, to encourage the community to advance technologies toward better screen content understanding. We anticipate that the advance of such technologies will benefit beyond just the screen UI and the human computer interaction (HCI) domains. As we will discuss in Section 3, other vision-language related 2This term is analogous to machine reading comprehension from natural language processing. 3ScreenQA dataset is released at https://github. com/google-research-datasets/screen_qa . 4Because questions are not exhaustively asked against a given screenshot.arXiv:2209.08199v2 [cs.CL] 22 Feb 2024multimodal domains share similar challenges with different emphases on respective modalities and contexts. Comparatively, ScreenQA is language and layout heavy, but it also includes visual ingredients such as icons and symbols as concise representations in place of texts, to declutter the UI. It may also include images or art designs that pose challenges to language centric machine learning agents. The remaining paper is organized in the follow- ing way: Section 2 formulates the problem, in- cluding the problem description and the evaluation metrics. We discuss relevant prior datasets and an- notations in Section 3 to put this work into perspec- tive. Section 4 describes our annotation method. The annotations are then analyzed in Section 5 to provide readers both the qualitative and quantita- tive views. The paper is concluded in Section 6 with a summary and a remark on future works. 2 Problem Setting We state the problem and define the evaluation metrics in this section. 2.1 Problem statement The ScreenQA task requires an agent to answer a user’s question by selecting relevant UI elements from a given single screenshot. When it comes with multiple relevant UI elements, a list of such UI elements whose contents minimally satisfy the question should be selected and ranked in descend- ing order of relevance to the question, if applicable, or following the common reading order by seman- tic groups, as will be described in Section 2.2. This assumes that answers are directly selectable from the screen and logical reasoning and calculation are not needed. If the screenshot does not contain the answers to the question, the agent should respond with “<no answer>”. This is summarized in Task 1. 2.2 Properties and terminologies The mobile app UI comes with some nuances. It is worth mentioning a few properties below. •View hierarchy, or the structural representa- tion used to render the screen, is not required in Task 1, to be consistent with the human an- notation process in Section 4. View hierarchy usually provides useful UI element candidates, but it may not always be reliable, for example, when using WebView or screen overlays. In such cases, a human annotator can still answerTask 1 ScreenQA Input: a question Qand a screenshot S Output: an answer list Aof UI elements selected fromSsuch that their contents minimally satisfy Q. The order of Ais further required to be •Ranked in descending order of relevance to Q, if applicable. •Otherwise, following the common reading or- der by semantic groups. If no contents in Scan satisfy Q, then returns an empty list A. screen questions entirely from pixels without an issue, so we want to benchmark similarly. We leave the choice of dependency on view hierarchies to the modelers and, hence, do not require it. However, this comes with an ambi- guity for UI element boundaries. See an ex- ample in Figure 1a. We devise a more flexible answer matching process to mitigate such an impact, as will be discussed in Section 2.3.3. •Avoid question answering over long para- graphs. Although it is permissive by Task 1, we discourage annotators from asking such questions during the annotation process. For ScreenQA, we want to focus on learning the relationships between text segments arranged two-dimensionally on the screen, and leave the long paragraph question answering, which investigates the relationships between words, to the traditional NLP domain. •Avoid logical reasoning. This task assumes answers can directly be extracted from the screenshot without reasoning, entailment, counting and comparing numbers. This fur- ther exclude yes/no and why questions if not explicitly displayed on the screen. The rea- son is that we want to separate “able to read” and “able to reason” and focus on the former first without generating an over challenging dataset. A few such excluded examples are: counting items, asking about the weather a few days from now, what are the items cheaper than X dollars, etc. •Ordered by relevance. The task is designed to enable the eyes-free user experience. That is, a user may not be fully aware of how many(a) Ambiguous UI element boundaries. Three possibilities are annotated. (b) Answers to “What’s the tempera- ture on Saturday?” (c) Semantic groups are the basic unit for ordering, not element coordinates. Figure 1: ScreenQA examples. (a) Determining UI elements can be sometimes ambiguous. Yet, a human can still answer screen questions without the aid of its view hierarchy. Hence, we do not require the view hierarchy as the input to the task, instead, employing a more flexible evaluation metric to mitigate the impact of this ambiguity, introduced in Section 2.3. (b) When asking a question, a user is assumed not knowing how many answers are relevant on the screen. All relevant information should be included and ordered by relevance and the common reading order. (c) Sometimes answers come as groups. In this case, “Deep Squat, 3 sets, 15x” should be grouped together and, by the reading order, followed by “Lunge, 3 sets, 10x” when being asked about the first two movement of deep squat. Simply following UI element coordinates would render the answer incomprehensible, e.g., “Deep Squat, Lunge, 3 sets, 15x, 3 sets, 10x”. Use semantic groups as the ordering unit instead. relevant answers are displayed on the screen. For example, in Figure 1b, when a user asks “What’s the temperature on Saturday?”, there are actually two temperatures, high and low, for each day and two Saturdays on the screen. In this case, the two temperatures should just follow the reading order, and the two Satur- days follow the relevance order as a user usu- ally refers to the upcoming Saturday. For a well-designed mobile app, these two usually overlap well and we do not expect a large am- biguity here. •Reading order by semantic groups. Some- times some UI elements are designed as se- mantic groups and should be referred to to- gether to keep their semantic meaning. For ex- ample, in Figure 1c, when a user asks “What are the first two movements of deep squat?”, then the answer should be “Deep Squat, 3 sets, 15x”, followed by “Lunge, 3 sets, 10x”. In other words, the common reading ordershould be based on semantic groups as the unit, rather than simply sorted by the coordi- nates of UI elements. Note that we set up the problem this way strategi- cally in order to prioritize its solvability consider- ing the progress of current technologies. However, practically, long vs. short texts and retrieval vs. rea- soning are naturally mixed together in the daily usage of mobile apps. We will leave this type of composite problems to the future work. 2.3 Evaluation metrics We consider two types of metrics: 1) Average normalized Discounted Cumulative Gain (Average nDCG) (Järvelin and Kekäläinen, 2002), which is commonly used in information retrieval and rank- ing systems, and 2) Average F1 score, which has been employed in closed-domain question answer- ing problems, such as the SQuAD dataset (Ra- jpurkar et al., 2016). One major difference between our metrics de-scribed below and the commonly used definitions is the unit of predictions. We use the element in the answer list A, described in Task 1, as the unit to determine a hit or a miss for both metrics. Be- sides, as UI elements can be ambiguous as men- tioned in Section 2.2, we will describe an answer matching algorithm that mitigate such an impact in Section 2.3.3. 2.3.1 Average nDCG We use a variant of nDCG that allows varying po- sitions (number of returns) as opposed to a typical fixed position. This is because, unlike the search problem, which is fair to evaluate, say, top-10 re- trieved documents across queries, ScreenQA can have different needs of answer lengths across differ- ent questions. For example, a question like “what is the address of the store” expects a single returned result. A question like “what are the login options?” expects an enumeration of options on the screen that easily go beyond five. Accordingly, we allow varying positions as follows: Given a 1-indexed listA, which is the predicted answer for the screen- question pair (S, Q), and a ground truth answer list Agfor(S, Q), the Discounted Cumulative Gain at varying positions (DCG v) is computed by: DCG v=∥A∥X i=1ri log2(i+ 1), (1) where ∥ · ∥ is the size of the list argument, riis the relevance score for the i-th item of A. We assign the relevance score 1 for a hit and 0 for a miss compared with the ground truth Ag. The corresponding Ideal Discounted Cumulative Gain (IDCG v) is computed by: IDCG v=∥Ag∥X i=11 log2(i+ 1). (2) The nDCG vis then nDCG v=DCG v IDCG v. (3) Note that nDCG vis still between 0 and 1, hence, convenient for comparing scores and computing the average. For a dataset of Nexamples, each of which is indexed by iand has a predicted answer Aiand Kground truth annotations Ag i,j=1...K, the averagenDCG vcan be computed by avg(nDCG v) =1 NNX i=1maxj[nDCG v(Ai, Ag i,j)]. (4) We choose a variant nDCG as the metric because 1) we want to measure the quality of the ranking. For example, if one incorrectly predicts the result from the first to the third position, the discount fac- tor brings down the score from 1.0 to only 0.5. 2) nDCG has an orthogonal design, which is easier to tweak toward a specific need than the mean aver- age precision (mAP) metric. For example, one can choose to discount faster or slower by changing the base of the denominator log2(i+ 1) and can choose to penalize irrelevant predictions by assign- ing negative scores. Mean reciprocal rank (MRR) and mAP are much less controllable in these two aspects. One known drawback of nDCG is that it does not naturally penalize excessive predictions after the last relevant item. We therefore use the average F1score as a complementary view of the agent performance. 2.3.2 Average F 1 Similar to the definition in SQuAD, the average F1score is computed as below, following the same notation as in (4): avg(F1) =1 NNX i=1maxj[F1(Ai, Ag i,j)].(5) Note that F 1does not concern ranking. For some cases, such as enumeration questions, this is de- sirable, as the ranking order is merely the reading order, even if the item order is permuted, the an- swer quality is in general not compromised, hence, reasonable to be assigned the same evaluation score. On the contrary, if relevance ranking is important, such as in Figure 1b, then nDCG provides a bet- ter view. Since both types of questions exist in the ScreenQA annotations, it is more complete to evaluate against both metrics. Also note that the unit of precision and recall computation is based on items in A, unlike SQuAD, which uses words as the unit instead. We describe how to compare items in an answer Awith the ground truth Agin the next section.2.3.3 Answer matching As mentioned in Section 2.2, the segmentation of UI elements provided in the predicted answer list Amay not coincide with the UI elements in the ground truth list Ag. Yet, if the overall answers are the same, the segmentation difference should not affect the evaluation score. Therefore, we use the following empirical procedure to mitigate such an impact, using an illustrated example (each capital- ized character is a word token): A= [”AB”,”B”,”BB”,”CBA ”] (6) Ag= [”AB”,”BC”,”AB”], (7) 1.Concatenate items in Ainto a single item list Ac= [“ABBBBCBA ”]. 2.Iterate through each g∈Agand check if g is contained in any item in Ac. If so, mark thegas HIT ( ✓) and mark the correspond- ing matched word token in the original Aand remove the matched part and split the remain- ing parts in Ac. Otherwise, mark the gas MISS ( ✗). In this example, when g= ”AB”, it is a HIT: A= [“A✓B✓”,“B”,“BB”,“CBA ”] Ac= [“BBBCBA ”] Ag= [“AB”✓,“BC”,“AB”]. Then when g= “BC”, it is a HIT. Note that the item in Acis split into two because of matching in the middle: A= [“A✓B✓”,“B”,“BB✓”,“C✓BA”] Ac= [“BB”,“BA”] Ag= [“AB”✓,“BC”✓,“AB”]. Last, when g= “AB”again, it is a MISS, A andAcunchanged, hence, omitted: Ag= [“AB”✓,“BC”✓,“AB”✗]. 3.Finally, iterate through each a∈A. If any a has at least one word token marked as HIT, then the whole ais a HIT, otherwise, a MISS. A= [“AB”✓,“B”✗,“BB”✓,“CBA ”✓]. This procedure converts AandAginto lists of HITs and MISSes. Then the evaluation metrics in (4) and (5) can be applied. Note that this procedure isnot order invariant. This in turn makes the F 1score not entirely order independent if any UI element ambiguity happens. This choice is to avoid the permutation complexity in evaluation. In practice, this is rarely an issue because when the ambiguity happens, the UI elements involved are almost al- ways tightly close to each other, making their order practically fixed. See Case 3 in Figure 1a as an example. 3 Related Datasets and Annotations ScreenQA has two aspects: multimodality and question answering. We discuss related problems and datasets from these two aspects and focus our survey on datasets that are 1) human annotated and 2) released to the public domain. 3.1 Multimodality Mobile app screenshots contain nearly all possi- ble representation of information through pixels. Most commonly, the information is majorly by text, blended with icons, symbols, and images.5 We discuss three related multimodal domains. 3.1.1 Screen UI for mobile apps For data released in the public domain, the RICO dataset (Deka et al., 2017) is, to the best of our knowledge, still the largest collection of mobile app screenshots (Deka et al., 2021). It contains 66k unique screenshots and their corresponding view hierarchies from 9.7k Android apps spanning 27 app categories. Its overall approach extended ER- ICA (Deka et al., 2016), which is an interactive trace recording tool and also released 3k traces for 18k unique screenshots from 1k Android apps for the search intent. LabelDroid (Chen et al., 2020b) and (Chen et al., 2020a) by the same authors released a dataset of 55k UI screenshots from 25 categories of 7.7k top-downloaded Android apps. Annotations and the corresponding problems can be roughly categorized by the scope of the contexts. At the UI element level, (Sunkara et al., 2022) anno- tated 77 icon types by shape, 15 out of which are ad- ditionally annotated with 38 semantic types, reach- ing about total 500k unique annotations. This work is further concerned with how UI elements are asso- ciated with companion labels such that the screen understanding between UI elements can be estab- lished. CLAY (Li et al., 2022) attempted to resolve 5Also videos, if we consider consecutive screenshots. We leave out the video modality here in the context of annotating the underlying RICO screenshots.the layout and view hierarchy denoising problem, annotating 60k RICO screenshots, a total of 1.4M UI elements with bounding boxes and types. (Li et al., 2020c) annotated 163k free-from descrip- tions for 61k UI elements from 22k RICO screen- shots. At the single-screen level, (Wang et al., 2021) collected text summarizations for screens, consisting of 112k screen descriptions across 22k RICO screenshots. At the multi-screen level, one challenging direc- tion is screen navigation, which requires the under- standing of screen states, feasible action spaces of the current screen, and overall task goals. Since multiple types of understandings are involved, this problem is not strictly focused on screen content understanding. PixelHelp (Li et al., 2020a) con- tains 187 multi-step instructions over 780 screen- shots for four task categories. MoTIF (Burns et al., 2022) contains 6k fine-grained instructions mixed with infeasible ones, over for 125 apps spanning 15 app categories. From the data perspective, anno- tating this type of problem is labor intensive and usually does not scale well. In comparison, the ScreenQA dataset is single- screen, focused on screen contents, and based on the RICO screenshots. 3.1.2 Document image understanding Document image understanding6concerns un- derstanding documents represented in pixels or scanned, photographed formats. This domain is similar to mobile app screens for its text-heavy and non-sequential nature. The most noticeable dataset is RVL-CDIP (Harley et al., 2015), a 400k-image subset from IIT-CDIP (Lewis et al., 2006), a collec- tion of low-resolution noisy documents, with bal- anced 16 document-level classes. FUNSD (Jaume et al., 2019) extracted a 199 scanned form im- ages from RVL-CDIP and annotated them with bounding boxes and 4 text-segment-level classes. SROIE (Huang et al., 2019) has 1k scanned receipt images for text localization, OCR, and key infor- mation extraction of 4 entity types. CORD (Park et al., 2019) contains 11k scanned receipt images, annotated with 9 classes and 54 subclasses for text segments in OCR boxes. These earlier works are more about classification for text segments or for the whole document image. A more recent work, DocVQA (Mathew et al., 2021), uses a question answering format for 6Also referred to as document analysis and recognition (DAR) or simply document understanding.span/segment extraction, with an annotation of 50k questions over 12k rectified, higher resolution doc- ument images. DocVQA is highly related to ScreenQA for its 2D arrangement of texts and for its extractive question answering format. We be- lieve that the techniques developed for screens and document images are cross applicable. 3.1.3 Visual question answering Visual question answering (VQA) (Antol et al., 2015) and screen UI are oftentimes mentioned to- gether, especially in the latter community, because of their vision-language multimodal nature. How- ever, VQA is distinctively different from screen understanding for two reasons: 1) The visual con- text for VQA is usually light in, or even free from, any text, while screen UI is the opposite, and 2) The images for VQA are typically photos of natural or daily scenes with objects, while screen UIs are in- formation oriented and arranged in a certain visual structure. There are some VQA variants compara- tively closer to screen UI, to mention a few: VQA for texts on objects in photos, e.g., VizWiz (Gurari et al., 2018) and TextVQA (Singh et al., 2019), and VQA for figures and charts, e.g., DVQA (Kafle et al., 2018), FigureQA (Kahou et al., 2018), and LEAF-QA (Chaudhry et al., 2020). These VQA tasks may appear as part of screens but in general are different problems. 3.2 Question answering Question answering tasks can be categorized by 1) open- or closed-domain, 2) answer formats and 3) main capacities to evaluate.7The com- mon answer formats include span (Rajpurkar et al., 2016), entity (Talmor and Berant, 2018), multi- ple choice (Mihaylov et al., 2018), and genera- tion (Xiong et al., 2019). The capacities to evaluate range from reading comprehension (Yang et al., 2015), multi-hop reasoning (Yang et al., 2018; Chen et al., 2021), logic reasoning (Yu et al., 2020), and commonsense reasoning (Talmor et al., 2019). From this question answering perspective, ScreenQA is a closed-domain question answering task that expects answers by span (or UI element phrase) selection for screen reading comprehen- sion. As described in Section 2, we instructed the data annotators to avoid multi-hop, mathematical counting, and logic reasoning, in order to focus on the fundamental screen comprehension capacity. 7Here we only include one or two examples per format and per capacity. This is by no means to be comprehensive.4 Annotation Method Figure 2: ScreenQA annotation process. We perform several steps to collect the ScreenQA annotations, as depicted in Figure 2. Each step is described below. 4.1 Pre-filtering The pre-filtering stage filters out 1) screenshots from non-English apps8, and 2) screenshots whose view hierarchies (VHs) are out of sync with the main contents. It is a known issue that in the RICO dataset, some screenshots and their corresponding view hierarchies are not perfectly in sync: there ex- ists certain time difference between view hierarchy extraction and screenshot capturing. We want to re- move those screenshots to ensure that all ScreenQA annotations are not subject to such data noises. Classifying the sync quality is tricky, even for hu- man readers. One may not be able to differentiate between occlusion, ghosting, and the actual out-of- sync. See Figure 3 for examples. Accordingly, we instructed the annotators to focus on the main con- tent area of the screen and make sure the bounding boxes in that area are not corrupted, as this is where most contents of interest and questions come from. We use 27 annotators to perform this step. Among RICO’s 66k unique screenshots, about 11k screenshots are from non-English apps, and about 13k screenshots have out-of-sync view hi- erarchies.9With the union of these two filtered out, there remains about 51k screenshots from English apps with in-sync VHs. 8This is different from “non-English screenshots”, as trans- lation and dictionary apps could pose confusion. 9This out-of-sync number is different from (Li et al., 2020b) because we focus on the main content area.4.2 Question annotations For question annotation, we asked the annotators to frame questions given a screenshot as the con- text. The annotators were expected to compose 1) natural, daily-life questions as if using the app. 2) The composed questions should inquire infor- mation that can directly read off from the screen and 3) should not require logical reasoning, count- ing and calculation, mathematical comparison, etc. We further required the annotators 4) not to ask questions about any advertisement on the screen. The annotation UI is depicted in Appendix A.1. We asked the annotators to compose up to five questions given a screenshot in the first pass. In the second pass, we asked for up to three questions given a screenshot and the questions previously composed. Each pass involved one annotator for each screenshot and whoever annotated the screen- shot before is excluded from being assigned to the same screenshot. This ensures that every screen- shot is assigned precisely two annotators to com- pose questions. We chose this sequential process 1) to avoid tricky deduplication of similar ques- tions, and 2) to encourage annotators to diversify their questions. Note that the same set of anno- tators were involved in the both passes such that each annotator had an opportunity to develop its own question style in the first pass before seeing others’ in the second pass. This makes sure that we still have certain numbers of question styles in the dataset before they converge to each other in repeated passes. We again involved the 27 annotators. The first pass of question annotation generated 46k ques- tions. The second pass added additional 36k ques- tions. These amount to a total of 82k questions, leaving about 15k screenshots with no questions annotated, due to lack of interesting contents. 4.3 Answer annotations We used the total 82k questions of 35k screenshots from the previous two-pass question annotation step to further annotate the corresponding answers. The annotator who composed the question is ex- cluded from annotating its own answer to avoid po- tential biases. The answer annotation UI is shown in Appendix A.2. Given an example, which contains a screenshot and a question, the annotators are tasked to 1.Fix any grammatical errors or typos of the given question without altering its intention.(a) In-sync VH with occluded UI ele- ments. (b) In-sync VH for main content, with ghosting VH from hamburger menu. (c) Out-of-sync VH for main content, though top bar VH is in-sync. Figure 3: View hierarchies (VHs) are overlaid on the screenshots with class names and the first few characters printed to assist annotators to determine whether the VHs for the main contents are in sync. a) The VH of some UI elements may be occluded. b) The VH of the hamburger menu is ghosting. Since the VH for the main content stays intact, data annotators are instructed to mark this as in-sync. c) Although the app bar and the orange plus button are in sync, the bounding boxes in the main content area are all out of sync. This is instructed to mark as out-of-sync and are excluded from further annotations for questions and answers. 2.Answer the question, based on the context of the given screenshot, by 1) selecting bounding boxes from the underlying view hierarchy leaf nodes that contain the relevant answers, or drawing bounding boxes if no suitable leaf nodes can be used, and 2) ranking the answers in descending order of relevance if applicable, or by the common reading order. 3.Additionally also provide a full-sentence an- swer to the question. 4.Consider two exceptions: 1) The question may be incomprehensible or 2) the screenshot does not contain the answer to the question, due to the questioner’s lack of understanding of the app. Then the example should be marked as “invalid question” and “not answerable from the screenshot”, respectively. 5.One answer is annotated for the train split, and three for the validation and the test splits. This is to improve the evaluation quality. The data split details will be described in Section 4.5.The “invalid question” annotations are then fil- tered out, and the questions that have no other answer annotations are excluded from the over- all ScreenQA dataset, as they are considered in- correctly annotated during the question annotation phase. 4.4 Not-answerable question annotations Figure 4: Chart showing the fraction of questions with answers and not answerable. Note that validation and test splits on average have roughly 3 answers per ques- tion, so there are cases when some annotators consid- ered a question to be not answerable, while others pro- vided an answer to that same question. Specifically, the validation and the test splits have 2.18% and 3.23% of such questions (the segments in red).(a) Question: ‘ What is the name of the application? ’ (b) Question: ‘ What is the contact email for tech support? ’ (c) Question: ‘ What is the date of ver- sion 1.3.1? ’ Figure 5: Examples of not answerable questions. The questions marked as “not answerable from the screenshot” represent a special category of ques- tions that check model overtriggering (attempting to answer those which are not supposed to be an- swered). Being able to come to a conclusion that the answer is not present on the screen is an impor- tant aspect of screen understanding. Note that it is possible that one annotator considered a question to be not answerable, and another provided an answer to that same question. As described in Section 4.2, the first two passes of question annotations aimed to compose ques- tions that can be answered from the screen, so as expected, the fraction of not answerable questions was small. We then had a third pass of question annotation to raise this fraction to nearly 10%, see Figure 4. For this, we used nearly 5k screenshots selected randomly from those where there were no such questions yet. In this pass, we asked annota- tors for exactly one additional question per screen- shot that had some relation to the information there, but could not be answered. See examples in Fig- ure 5. Answer annotation was not used for these 5k questions. 4.5 Dataset statistics The ScreenQA dataset contains 35,352 screenshots and 86,025 questions.Screenshots Questions Train 28,378 68 ,980 Validation 3,485 8 ,618 Test 3,489 8 ,427 Total 35,352 86 ,025 Table 1: ScreenQA dataset split stats. It is split into train, validation and test sets in approximately 80-10-10 ratio, see Table 1. Note that all questions for the same screenshot belong to only one split. 5 Annotation Analysis We analyze the annotations of questions and an- swers in this section. 5.1 Question analysis We collected overall 86k questions over 35k unique screenshots from RICO. Among the 86k questions, there are 47.5k unique questions.10Some screen- shots receive more questions because they usually contain more information to be asked about. Yet, 10Note that it is natural and valid to ask the same com- mon questions over various screenshots, for example, “Which option is selected on the screen?” and “What is the email address?”Category % Examples UI selection & config 18.1 Which option is selected? What is the selected ringtone? Quantity number 11.7 How many unread messages? How many pictures are there in Western Europe? App name 10.4 What is the name of the application? What is the app name? Date time 9.4 When was “Heal the Living” released? When is happy hour? Price 3.4 How much is the gift bonus in 3rd place? What is the price? Name of item 3.3 What is the name of the drug? What is the name of chef? User name 2.8 What is the name of the user? What is the username on telegram? Duration 2.5 What is the duration of video? How long is the song? Enum. of avail. options 2.5 Which social media options are given there? What are the options available for logging in? Address and direction 2.4 What is the current location? What is the service zip code? Email address 2.4 What is an email address? What is customer service email? Person’s name 2.1 Who sang the song? What is the last name? Signup/login 1.6 Which application can be used to sign up / login? What are the alternative choices for signing up? Version information 1.6 What is the version number? What is the new feature in version v3.1.3? Weather 1.5 What is the range of temperature shown on Sunday? What is the weather forecast for Sunday? Score & value 1.4 What is height/weight of the person? What is the score? Yes/No 1.1 Is there any travel plans? Is there any favorite? Phone number 1.0 What is the phone number? What is the prefix for the international mobile number? # of Stars 0.8 What is the star rating? How many stars are given to the product? Share/sharing 0.8 Which application can be used to share? Where can I share this application? Age 0.8 How old is ...? What is the age? Percentage 0.7 What is the percentage of ... ? What is the brightness percentage for foreground? Settings 0.6 What is the setting of ... ? Which settings are switched on? Quantity amount 0.6 How much fat is there? What is the amount? Permission 0.5 Which application is asking for permissions? What permissions are required for MyCarTracks? # of Likes 0.5 How many likes for ... ? How many likes does ... get? Country 0.5 What is the name of the country? Which country has the +54 code? Distance 0.5 What is the visibility distance? How far is it from ... ? # of Reviews 0.4 What is the number of comments on ... ? How many comments? Website 0.3 What is the url? What’s the website address? Gender 0.3 What is the gender? Which gender is displayed on the screen? How to 0.3 How to start on boot? How to pronounce his name? Currency 0.3 What is the currency? What is the currency for the price? Unit of measurement 0.2 What is the unit of temperature? What is the unit of weight and length? Language 0.1 Which language is used in the setting? Which language is being translated into which language? Color 0.0 What is the UI color? What is the amount of green color? Others 12.8 What’s the average speed? What is the user’s middle initial What is the spending limit? Which team has 41 points? Total 100.0 Table 2: Question category distribution and examples. the histogram still exhibits a reasonable exponential decay with a mild slope, as depicted in Figure 6a. To further understand what questions have been asked, we categorize the questions using regular ex- pressions based on a list of empirically determined question categories. The categories are meant to provide a rough overview of the question annota- tions and by no means to provide a precise catego- rization. The distribution and examples by these categories are tabulated in Table 2. Note that the questions were not composed at the annotators’ full discretion: They are conditioned on the given screenshots. That is to say, the distribution is im- plicitly influenced by the RICO crawling process. For example, as RICO crawled screen traces from freshly installed apps and did not login an account, a noticeable number of the screen traces end at a login page. This in turn translates to a higher per- centage of questions asked about app names, email addresses, permissions to login, etc.5.2 Answer analysis We analyze the answer annotations in two as- pects: 1) How often do we need more than one bounding box and its text to answer the question, and 2) How often do human annotators find the view hierarchy useful to provide a minimal answer to the question. Figure 6b illustrates the histogram of number of bounding boxes used in each answer. About 84% of answers contain a single bounding box. Among these single-bounding-box answers, 51% uses a VH leaf node directly, while 49% uses a manually drawn bounding box. If we consider all answers together, excluding cases when there is no answer, still 51% uses VH leaf nodes entirely, while 48% uses manually drawn bounding boxes. That is, for about half of the total number of screenshots, human annotators preferred to manually draw the bounding boxes in order to provide answers that minimally satisfy the question. This observation re- flects the necessity not to require the view hierarchy input for ScreenQA as described in Task 1. Interestingly, there exist some cases, about 0.8%(a) Number of composed questions per screenshot. (b) Number of bounding boxes used to answer the question. Figure 6: Histograms for number of composed questions and number of bounding boxes in answers. a) The three question annotation passes were capped at five, three and one questions, respectively, resulting in the maximum nine questions in total. b) The cases when there is no answer or a single bounding box forms a sufficient answer amount to 91-92% of the answers, hence removed from the chart for the clarity of the long tail. Anything beyond 10 bounding boxes is less than 0.07%, accumulatively less than 0.15%. of the questions, that the human annotators used a mixture of VH leaf nodes and manually drawn bounding boxes as their full answer. By inspecting those cases, we found that these usually happen 1) when the answer is an enumeration of “inhomo- geneous” options that are organized differently on the screen, such as using email vs. other APIs to login, and 2) when an answer needs multiple parts to be complete, such as a date consisting of year, month, and day scattered on the calendar UI, and a temperature or a measurement requiring a number followed by the corresponding unit. These parts may not be displayed in the same way, resulting in lack of useful VH leaf nodes for some of the parts. 6 Conclusion In this work, we proposed the ScreenQA task. We annotated a large-scale ScreenQA dataset, which contains 86,025 question-answer pairs. Compared to other vision-language multimodal problems, such as document image understanding and visual question answering, ScreenQA poses its unique challenges: rich in text, diverse in apps, and blended with icons and symbols. We hope to use the ScreenQA task and the dataset to encourage the community to look into this screen content under- standing problem, as it enables new technologies and new user experiences. 7 Acknowledgements The authors would like to thank Srinivas Sunkara for his valuable discussions and comments on this manuscript.References Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Mar- garet Mitchell, Dhruv Batra, C. Lawrence Zitnick, and Devi Parikh. 2015. VQA: Visual Question An- swering. In 2015 IEEE International Conference on Computer Vision (ICCV) , pages 2425–2433, Santi- ago, Chile. IEEE. Andrea Burns, Deniz Arsan, Sanjna Agrawal, Ranjitha Kumar, Kate Saenko, and Bryan A. Plummer. 2022. A Dataset for Interactive Vision-Language Naviga- tion with Unknown Command Feasibility. Ritwick Chaudhry, Sumit Shekhar, Utkarsh Gupta, Pranav Maneriker, Prann Bansal, and Ajay Joshi. 2020. LEAF-QA: Locate, Encode & Attend for Figure Question Answering. In 2020 IEEE Win- ter Conference on Applications of Computer Vision (WACV) , pages 3501–3510, Snowmass Village, CO, USA. IEEE. Jieshan Chen, Chunyang Chen, Zhenchang Xing, Xin Xia, Liming Zhu, John Grundy, and Jinshui Wang. 2020a. Wireframe-based UI Design Search through Image Autoencoder. ACM Transactions on Software Engineering and Methodology , 29(3):19:1–19:31. Jieshan Chen, Chunyang Chen, Zhenchang Xing, Xi- wei Xu, Liming Zhu, Guoqiang Li, and Jinshui Wang. 2020b. Unblind your apps: Predicting natural- language labels for mobile GUI components by deep learning. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering , ICSE ’20, pages 322–334, New York, NY , USA. As- sociation for Computing Machinery. Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, and William Yang Wang. 2021. FinQA: A Dataset of Nu- merical Reasoning over Financial Data. In Proceed- ings of the 2021 Conference on Empirical Methodsin Natural Language Processing , pages 3697–3711, Online and Punta Cana, Dominican Republic. Asso- ciation for Computational Linguistics. Biplab Deka, Bardia Doosti, Forrest Huang, Chad Franzen, Joshua Hibschman, Daniel Afergan, Yang Li, Ranjitha Kumar, Tao Dong, and Jeffrey Nichols. 2021. An Early Rico Retrospective: Three Years of Uses for a Mobile App Dataset. In Yang Li and Otmar Hilliges, editors, Artificial Intelligence for Human Computer Interaction: A Modern Approach , pages 229–256. Springer International Publishing, Cham. Biplab Deka, Zifeng Huang, Chad Franzen, Joshua Hi- bschman, Daniel Afergan, Yang Li, Jeffrey Nichols, and Ranjitha Kumar. 2017. Rico: A Mobile App Dataset for Building Data-Driven Design Applica- tions. In Proceedings of the 30th Annual ACM Sym- posium on User Interface Software and Technology , UIST ’17, pages 845–854, New York, NY , USA. As- sociation for Computing Machinery. Biplab Deka, Zifeng Huang, and Ranjitha Kumar. 2016. ERICA: Interaction Mining Mobile Apps. In Pro- ceedings of the 29th Annual Symposium on User Interface Software and Technology , UIST ’16, pages 767–776, New York, NY , USA. Association for Com- puting Machinery. Danna Gurari, Qing Li, Abigale J. Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P. Bigham. 2018. VizWiz Grand Challenge: Answer- ing Visual Questions from Blind People. In 2018 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition , pages 3608–3617, Salt Lake City, UT, USA. IEEE. Adam W. Harley, Alex Ufkes, and Konstantinos G. Der- panis. 2015. Evaluation of deep convolutional nets for document image classification and retrieval. In 2015 13th International Conference on Document Analysis and Recognition (ICDAR) , pages 991–995. IEEE Computer Society. Zheng Huang, Kai Chen, Jianhua He, Xiang Bai, Dimos- thenis Karatzas, Shjian Lu, and C. V . Jawahar. 2019. ICDAR2019 Competition on Scanned Receipt OCR and Information Extraction. In 2019 International Conference on Document Analysis and Recognition (ICDAR) , pages 1516–1520. Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumu- lated gain-based evaluation of IR techniques. ACM Transactions on Information Systems , 20(4):422– 446. Guillaume Jaume, Hazim Kemal Ekenel, and Jean- Philippe Thiran. 2019. FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents. arXiv:1905.13538 [cs, stat] . Kushal Kafle, Brian Price, Scott Cohen, and Christopher Kanan. 2018. DVQA: Understanding Data Visualiza- tions via Question Answering. In 2018 IEEE/CVFConference on Computer Vision and Pattern Recog- nition , pages 5648–5656, Salt Lake City, UT. IEEE. Samira Ebrahimi Kahou, Vincent Michalski, Adam Atkinson, Ákos Kádár, Adam Trischler, and Yoshua Bengio. 2018. FigureQA: An Annotated Figure Dataset For Visual Reasoning. In International Conference on Learning Representations (ICLR) , page 20. D. Lewis, G. Agam, S. Argamon, O. Frieder, D. Gross- man, and J. Heard. 2006. Building a test collection for complex document information processing. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval , SIGIR ’06, pages 665–666, New York, NY , USA. Association for Computing Machinery. Gang Li, Gilles Baechler, Manuel Tragut, and Yang Li. 2022. Learning to Denoise Raw Mobile UI Layouts for Improving Datasets at Scale. In CHI Conference on Human Factors in Computing Systems , pages 1– 13, New Orleans LA USA. ACM. Yang Li, Jiacong He, Xin Zhou, Yuan Zhang, and Ja- son Baldridge. 2020a. Mapping Natural Language Instructions to Mobile UI Action Sequences. In Pro- ceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics , pages 8198– 8210, Online. Association for Computational Lin- guistics. Yang Li, Jiacong He, Xin Zhou, Yuan Zhang, and Ja- son Baldridge. 2020b. Mapping Natural Language Instructions to Mobile UI Action Sequences. In Pro- ceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics , pages 8198– 8210, Online. Association for Computational Lin- guistics. Yang Li, Gang Li, Luheng He, Jingjie Zheng, Hong Li, and Zhiwei Guan. 2020c. Widget Captioning: Generating Natural Language Description for Mo- bile User Interface Elements. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages 5495–5510, Online. Association for Computational Linguistics. Minesh Mathew, Dimosthenis Karatzas, and C. V . Jawa- har. 2021. DocVQA: A Dataset for VQA on Doc- ument Images. In 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) , pages 2199–2208, Waikoloa, HI, USA. IEEE. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a Suit of Armor Conduct Elec- tricity? A New Dataset for Open Book Question An- swering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pages 2381–2391, Brussels, Belgium. Association for Computational Linguistics. Seunghyun Park, Seung Shin, Bado Lee, Junyeop Lee, Jaeheung Surh, Minjoon Seo, and Hwalsuk Lee. 2019.CORD: A Consolidated Receipt Dataset for Post- OCR Parsing. In Workshop on Document Intelli- gence at NeurIPS 2019 . Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ Questions for Machine Comprehension of Text. In Proceedings of the 2016 Conference on Empirical Methods in Natu- ral Language Processing , pages 2383–2392, Austin, Texas. Association for Computational Linguistics. Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. 2019. Towards VQA Models That Can Read. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 8309–8318, Long Beach, CA, USA. IEEE. Srinivas K. Sunkara, Maria Wang, Lijuan Liu, Gilles Baechler, Yu-Chung Hsiao, Jindong Chen, Abhanshu Sharma, and James W. Stout. 2022. Towards Better Semantic Understanding of Mobile Interfaces. In Proceedings of the 30th International Conference on Computational Linguistics , Gyeongju, Republic of Korea. International Committee on Computational Linguistics. Alon Talmor and Jonathan Berant. 2018. The Web as a Knowledge-Base for Answering Complex Ques- tions. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technolo- gies, Volume 1 (Long Papers) , pages 641–651, New Orleans, Louisiana. Association for Computational Linguistics. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. CommonsenseQA: A Ques- tion Answering Challenge Targeting Commonsense Knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume 1 (Long and Short Papers) , pages 4149–4158, Minneapolis, Minnesota. Association for Computational Linguistics. Bryan Wang, Gang Li, Xin Zhou, Zhourong Chen, Tovi Grossman, and Yang Li. 2021. Screen2Words: Au- tomatic Mobile UI Summarization with Multimodal Learning. In The 34th Annual ACM Symposium on User Interface Software and Technology , UIST ’21, pages 498–510, New York, NY , USA. Association for Computing Machinery. Wenhan Xiong, Jiawei Wu, Hong Wang, Vivek Kulka- rni, Mo Yu, Shiyu Chang, Xiaoxiao Guo, and William Yang Wang. 2019. TWEETQA: A Social Media Focused Question Answering Dataset. In Pro- ceedings of the 57th Annual Meeting of the Asso- ciation for Computational Linguistics , pages 5020– 5031, Florence, Italy. Association for Computational Linguistics. Yi Yang, Wen-tau Yih, and Christopher Meek. 2015. WikiQA: A Challenge Dataset for Open-DomainQuestion Answering. In Proceedings of the 2015 Conference on Empirical Methods in Natural Lan- guage Processing , pages 2013–2018, Lisbon, Portu- gal. Association for Computational Linguistics. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christo- pher D. Manning. 2018. HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. InProceedings of the 2018 Conference on Empiri- cal Methods in Natural Language Processing , pages 2369–2380, Brussels, Belgium. Association for Com- putational Linguistics. Weihao Yu, Zihang Jiang, Yanfei Dong, and Jiashi Feng. 2020. ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning. In International Con- ference on Learning Representations . AData annotation interfaces for question and answer collection A.1 Question annotation interface The question annotation interface is shown in Fig- ure 7. Question annotation was performed in a sequential manner, the later and non-overlapping annotators can see all previous questions to diver- sify question framing and avoid duplication. We also used the sequential process to provide more feedback and training to the annotators for quality improvement. A.2 Answer annotation interface The answer annotation interface is shown in Fig- ure 8. Answer annotators were tasked to determine if the question is valid and if the question is an- swerable from the screen context. If both are posi- tive, the annotators need to answer the questions by 1) selecting or drawing the bounding boxes of UI elements, 2) filling the text for each selected/drawn bounding box on right right, and 3) ranking them appropriately. The annotators were also tasked to review and make necessary corrections if the ques- tion has grammatical errors or typos.Figure 7: Question annotation interface.Figure 8: Answer annotation interface.

Paper 63d637283e7807d2a72ce6d94ccb35c9edd76fed not found in buffer: ['410e4e70ec38f9a4cfd43df6dd92adc9f739ce2a', '40e3cd767c9... Paper 63d637283e7807d2a72ce6d94ccb35c9edd76fed not found in buffer: ['410e4e70ec38f9a4cfd43df6dd92adc9f739ce2a', '40e3cd767c9520171ac8ecbd2e27f3a1bf043b61', 'd777d15677b673fe02750333181ea5e1e6dadf17', 'fe351bc1bda19583a7d6b1a96a9b18f59c40b213', '441068ef17c3332d4f0af6c146e37bd8748bb90e', '781402fae3a7a44b90855eeba8e96a872347fd4b', '3a6933886485038805e7af23a38845f9d93e1b9c', 'c70cefca5c76724b27e4615259be889d9ca3aa92', '42d83576ee920c1b6df318e212047d9ba57fc4fd', '7fff3df60b8d9a5e57627f4eb6b3022b60f3a39d']. Please try a different paper. Paper 63d637283e7807d2a72ce6d94ccb35c9edd76fed not found in buffer: ['410e4e70ec38f9a4cfd43df6dd92adc9f739ce2a', '40e3cd767c9520171ac8ecbd2e27f3a1bf043b61', 'd777d15677b673fe02750333181ea5e1e6dadf17', 'fe351bc1bda19583a7d6b1a96a9b18f59c40b213', '441068ef17c3332d4f0af6c146e37bd8748bb90e', '781402fae3a7a44b90855eeba8e96a872347fd4b', '3a6933886485038805e7af23a38845f9d93e1b9c', 'c70cefca5c76724b27e4615259be889d9ca3aa92', '42d83576ee920c1b6df318e212047d9ba57fc4fd', '7fff3df60b8d9a5e57627f4eb6b3022b60f3a39d']. Please try a different paper.

Paper 8e01f640c39f974bddbcca957d1580f0dd47f511 not found in buffer: ['410e4e70ec38f9a4cfd43df6dd92adc9f739ce2a', '40e3cd767c9... Paper 8e01f640c39f974bddbcca957d1580f0dd47f511 not found in buffer: ['410e4e70ec38f9a4cfd43df6dd92adc9f739ce2a', '40e3cd767c9520171ac8ecbd2e27f3a1bf043b61', 'd777d15677b673fe02750333181ea5e1e6dadf17', 'fe351bc1bda19583a7d6b1a96a9b18f59c40b213', '441068ef17c3332d4f0af6c146e37bd8748bb90e', '781402fae3a7a44b90855eeba8e96a872347fd4b', '3a6933886485038805e7af23a38845f9d93e1b9c', 'c70cefca5c76724b27e4615259be889d9ca3aa92', '42d83576ee920c1b6df318e212047d9ba57fc4fd', '7fff3df60b8d9a5e57627f4eb6b3022b60f3a39d']. Please try a different paper. Paper 8e01f640c39f974bddbcca957d1580f0dd47f511 not found in buffer: ['410e4e70ec38f9a4cfd43df6dd92adc9f739ce2a', '40e3cd767c9520171ac8ecbd2e27f3a1bf043b61', 'd777d15677b673fe02750333181ea5e1e6dadf17', 'fe351bc1bda19583a7d6b1a96a9b18f59c40b213', '441068ef17c3332d4f0af6c146e37bd8748bb90e', '781402fae3a7a44b90855eeba8e96a872347fd4b', '3a6933886485038805e7af23a38845f9d93e1b9c', 'c70cefca5c76724b27e4615259be889d9ca3aa92', '42d83576ee920c1b6df318e212047d9ba57fc4fd', '7fff3df60b8d9a5e57627f4eb6b3022b60f3a39d']. Please try a different paper.

Can Incremental Learning help with KG Completion? Faculty of Media Engineering and Technology, German University in Cairo, Egypt Abstract. K... Can Incremental Learning help with KG Completion? Faculty of Media Engineering and Technology, German University in Cairo, Egypt Abstract. Knowledge Graphs (KGs) are a type of knowledge representation that gained a lot of attention due to their ability to store information in a structured format. This structure representation makes KGs naturally suited for search engines and NLP tasks like question-answering (QA) and task-oriented systems; however, KGs are hard to construct. While QA datasets are more available and easier to construct, they lack structural representation. This availability of QA datasets made them a rich resource for machine learning models, but these models benefit from the implicit structure in such datasets. We propose a framework to make this structure more pronounced and extract KG from QA datasets in an end-to-end manner, allowing the system to learn new knowledge in incremental learning with a human-in-the-loop (HITL) when needed. We test our framework using the... Can Incremental Learning help with KG Completion? Faculty of Media Engineering and Technology, German University in Cairo, Egypt Abstract. Knowledge Graphs (KGs) are a type of knowledge representation that gained a lot of attention due to their ability to store information in a structured format. This structure representation makes KGs naturally suited for search engines and NLP tasks like question-answering (QA) and task-oriented systems; however, KGs are hard to construct. While QA datasets are more available and easier to construct, they lack structural representation. This availability of QA datasets made them a rich resource for machine learning models, but these models benefit from the implicit structure in such datasets. We propose a framework to make this structure more pronounced and extract KG from QA datasets in an end-to-end manner, allowing the system to learn new knowledge in incremental learning with a human-in-the-loop (HITL) when needed. We test our framework using the SQuAD dataset and our incremental learning approach with two datasets, YAGO3-10 and FB15K237, both of which show promising results. Keywords: Knowledge Graphs, Question Answering, Incremental Learning, Human in the loop 1 Introduction Task-oriented dialogue systems have been a big part of our modern life and an active area for research and industry. They aim to chat with the user to understand their needs and achieve a specific task for them, and this could be an actual task like setting the alarm, answering a question, or recommending something for the user. These systems could work as multi-domain systems, covering more than one domain, or closed-domain systems, covering only a single domain. They could achieve the goal by conversing with the user over multiple or single turns. Task-oriented dialogue systems, in general, either follow a modular pipeline approach consisting of four main modules; the first one is a Natural Language Understanding (NLU) module: which is responsible for extracting the information from the user utterance into the knowledge format the model understands, the second one is a Dialogue State Tracking (DST) module: that is responsible for updating the current state of the dialogue, the third module is the Dialogue Policy (DP): which decides for the following action based on the current state, and the last one is a Natural Language Generator (NLG): that takes action decided upon and generates the response to the user in the form of natural language. Sometimes, the DST and DP are referred to as the Dialogue Manager. The other approach is for the system to work end-to-end.[1] How the knowledge is represented and understood affects the architecture and the techniques used to reach the objective goal. Consequently, many approaches were pro- posed in this area, which motivated the construction of many datasets. The knowledge representation of the training dataset affects the techniques used to process and extract the needed information. Some of the most common data representations are: –Questions and Answers pairs; are the most accessible form of knowledge to construct a dataset, as we only need to collect the previous logs of conversations when it’s available. –Documents; in which the knowledge is stored in paragraphs containing the information needed about the system.Mayar Osama Mervat Abu-Elkheir pp. 103-122, 2023. CS & IT - CSCP 2023 DOI: 10.5121/csit.2023.130510David C. Wyld et al. (Eds): CCSEA, AIFU, EMSA, NLCAI, NCOM, SIPRO, SEA, DKMP, BDML, BIoT, CLOUD - 2023–Intents and slots values representation is a catalog-like representation to store the infor- mation, where the intent detected from the user’s query is used to fill its corresponding slot. The slots are the labels each word token has in the text. –Knowledge Graphs, at which the knowledge is represented as facts, where each fact is a triplet (h, r, t) where the head entity his connected to the tail entity tby relation r. The most expensive and time-consuming forms of knowledge representation are knowl- edge graphs or intents and slot values; although they provide the best results since they are handcrafted, especially in closed domain systems, they require experts to tailor the knowledge with the required format. In comparison, having the knowledge represented as pairs of questions and answers is easier to be collected from existing systems’ logs. Same for having multi-paragraph documents, which are easier to construct. Constructing a new dataset is costly and time-consuming. Hence, we propose the in- tegration of existing datasets and trying to represent the same knowledge with different representation forms. This paper focuses on extracting knowledge from question-answer pairs and constructing a graph containing the equivalent knowledge. Knowledge graphs (KGs) represent the data formally as a set of facts; a fact triplet represents the relation between two entities, so these entities could be viewed as the nodes of the graph and the relations to be the edges connecting them. A fact is also referred to as a triplet (h, r, t), where the head entity h is connected to the tail entity t by relation r. Thanks to their structural representation, KGs are used in many NLP task-oriented tasks, e.g., information retrieval, search agents, question answering, conversational recommender systems, etc... As for every knowledge representation, KGs have limitations beyond being more ex- pensive to construct. The main task of extracting knowledge from KGs is a link prediction task. Link prediction is the task of predicting the missing entity in a given query depend- ing on whether it’s a (h,r,?) tail prediction or a (?,r,t) head prediction, the model should predict the missing entity to complete the required fact. This link prediction could be done straightforwardly or require multiple hops between a few facts in the KG until we reach the missing entity. The issue of not being able to retrieve the missing entity right away motivated another active task in the field of KGs, which is the task of reasoning over KGs. Reasoning over KGs aims to obtain new facts from existing knowledge. Chen et al. [2] reviewed over 11 different approaches for reasoning over KGs, which is an active task for KGs because KGs suffer the limitation of incompleteness. The incompleteness issue exists in most datasets as it’s hard to capture all the needed knowledge of a given topic, especially when designing an open-domain system; it’s almost impossible to cover everything when creating the knowledge base at the beginning. Al- though reasoning over KGs helps reduce that gap, all the data to be added would be at some point, and no new knowledge could be added. Hence, a dynamic system is needed to add new knowledge when needed. We address this issue by having a human in the loop. A human in the loop is used in different fields to achieve different tasks [3,4,5,6]. The idea behind it is to deploy the model and let it work as it should, and when the model needs new knowledge or makes a mistake, the human/expert could interfere and update the model with the needed knowledge. In our approach, we achieve this with an incremen- tal learning module. But having a human in the loop pops out an important question, does this contradict the main aim of artificial intelligence and machine learning to automate a given task and remove the human from the loop? And the answer is it’s a trade-off. For a particular closed-domain system, it is very doable to construct a complete knowledge base from the beginning that the model could be trained on and answer any given ques-104 Computer Science & Information Technology (CS & IT)tion accordingly. But in open-domain systems, constructing a complete knowledge base is almost impossible. And assuming there was a way to collect all the relevant knowledge from multiple sources would require intensive training, which would be very expensive. It might also be that not all the knowledge gathered is needed, which might backfire and affect the model’s performance. Hence, having a human in the loop only when needed is an acceptable middle ground in our approach, as we start by extracting the knowledge in an end-to-end manner and only refer to the human in the loop when needed to reduce the gap between the knowledge collected. Eventually, the dependency on the human in the loop decreases till it’s not needed anymore. This paper aims to address the following three issues; 1) Constructing KGs from question-answering datasets in an end-to-end manner, 2) Having a dynamic knowledge base system to learn new facts and allow the model to learn new entities, and 3) Reducing the gap of having 1:n or n:1 relations in the KG for existing KG embedding models. 2 Background and Related Work 2.1 Knowledge Graph Embedding and Link Prediction KGs could be formally represented as a set of facts, such that a fact consists of a triplet (e1, r,e2)or(head, relation, tail) ; i.e. the subject e1is connected to the object e2through this relation r, where e1ande2belong to the set of possible entities and r belongs to the set of possible relations. KG={(e1,r,e2) ∥e1,e2∈Eand r ∈R} Link prediction is the task of predicting the missing entity given a source entity and a relation. It could be a head prediction where the source entity is the tail entity, and the missing entity is the head (?,r,t) , or a tail prediction where the source entity is the head, and the missing entity is the tail (h,r,?) . Many approaches have been proposed to achieve this link prediction task, some focused on observable features such as Rule Mining [17][16][38][24] or the Path Ranking Algorithm [31][32], and others focused on capturing latent features of the graph by using different embedding techniques. In our paper, we are mainly focusing on the KG embedding ap- proaches. In general, how the task of link prediction works with KG embedding models is by defining a scoring function ϕthat indicates the probability of the given fact being true. As shown in Equation 1, for a given tail prediction, the model should output the entity e, which returns the highest score from the scoring function. t= arg max e∈Eϕ(h, r, e ) (1) Rossi et al.[7] provided a very useful comparative analysis for many of these link prediction approaches for KGs. They classified these models into three main categories: 1. Tensor Decomposition Models, where the task of LP is considered a tensor decom- position task, as these models process the KG as a 3D adjacency matrix or a 3-way tensor that is only partially observable due to the KG incompleteness. This tensor is then decomposed into low-dimensional vectors, which are used as the embeddings for entities and relations. (a) Bilinear Models: Given a head embedding h∈Rdand a tail embedding t∈Rd, these models usually represent the relation embedding as a bi-dimensional matrix r∈Rd×d, where the scoring function computes the result of the product of the three matrices ϕ(h, r, t ) =h×r×t. One of the most commonly used models of this class is ComplEX[8].Computer Science & Information Technology (CS & IT) 105(b) Non-Bilinear Models combine the head, relation, and tail embeddings of compo- sition with approaches different from the strictly bilinear product. e.g., HolE[9] computes the circular correlation between the embeddings of head and tail entities and then applies the matrix multiplication with the relation embedding. 2. Geometric Models, on the other hand, view relations as geometric transformations in the latent space where the fact score is represented as the distance between a resulting vector of processing the head and the relation and the tail vector. (a) Pure Translational Models represent entities and relations as one-dimensional vec- tors of the same length, where the added distance between the head embedding and relation embedding should result in the position closest to the tail embedding. TransE[10] was the first proposed model using a pure translational approach, and due to this nature in calculating the score, TransE cannot correctly handle one-to- many and many-to-one relations, as well as symmetric and transitive relations. (b) Translational models with Additional Embeddings may associate more than one embedding to each KG element. For instance, CrossE[11] is considered one of the best models of this class. CrossE learns an additional relation-specific embedding with each relation crthat is then combined with the head and the relation to be used in the translation. (c) Roto-Translational Models that perform rotation-like transformations either in combination or in alternative to translations, e.g., RotatE[12] represents relations as rotations in a complex latent space. 3. Deep Learning Models use deep neural network layers to extract the features from the input by fine-tuning the weights and the biases of the neurons of these layers along with learning the KG embeddings. (a) Convolutional Neural Networks may contain one or more convolutional layer(s). The task of those layers is to loop over the input with convolution techniques by applying low-dimensional filters to allow the model to extract the needed features during the training phase. Then a Dense layer is used to process the output of the convolution to get the score of a given fact. Examples of CNN models for KG embeddings are ConvE[13], ConvKB[14], and ConvR[15]. (b) Capsule Neural Networks consist of capsules that are composed of groups of neurons that aim to encode specific features of the input. The main difference is that cap- sules allow the model to encode those features without losing spatial information, unlike convolutional networks. e.g. CapsE[16]. (c) Recurrent Neural Networks (RNNs), which consist of recurrent layers that are known for their abilities to process and encode sequential data, e.g., RSN[17]. 2.2 Knowledge Graph Construction The task of constructing a knowledge graph is usually done by experts to ensure the right format and cover the correct information, which is a very time-consuming and expensive process. Hence, many approaches have been proposed to try and automate this process. Some approaches focused on the potential of language models, as they get to learn linguistic knowledge during training and their ability to store relational knowledge between the training data. Language models are known to have the ability to implicitly encode massive amounts of knowledge, to be used for different tasks like question-answering, text summarizing, etc. One of the main advantages when using language models is that they don’t require a fixed schema or human annotations, which allows them to support open- domain questions and the ability to extend to more data.106 Computer Science & Information Technology (CS & IT)Language models could answer queries that are structured as “fill in the blank” cloze statements because of their masking mechanism during training, at which the model is required to learn to fill the word at the masked position with the correct word. Petroni et al[18] provided an analysis for language models to test their ability on factual and commonsense knowledge, where the facts they used were either triplet subject-relation- object or question-answer pairs. Each fact is converted to a cloze statement which is then queried to the language model for a missing token and accordingly evaluated that model. They used Google-RE1, T-rex[19], ConceptNet[20], and SQuAD[21] datasets. Their results showed the potential of BERT, as it performed well on open-domain ques- tions, and results showed that it contains relational knowledge that could be competitive with traditional NLP methods. Traditional NLP methods are known to contain the best information extractions on fixed schema, which is not always available. Also, they work in a complex pipeline to achieve entity extraction, coreference resolution, entity linking, and relation extractions. The pipeline architecture makes them vulnerable to error propagation and accumulation. Another approach that took advantage of language models in KG construction is MAMA[22]. They argued that language models are considered an open knowledge graph, as having a language model and a textual corpus, they could generate a knowledge graph relevant to that corpora through a two-staged architecture; Match and Map. At the match phase, the model generates a set of candidate facts from the corpus using Beam Search, where the goal is to match the knowledge stored in the pre-trained language model with the facts in the corpus. During the map phase, the generated facts are mapped to a fixed and open schema to generate the final knowledge graph. MAMA was not the only pro- posed approach to generate KGs using language models. Swamy et al.[23] followed the same property in language models to answer “fill in the blank queries” cloze statements as LAMA[18]. They first used this property to extract all statements that contain rele- vant knowledge using the masking property, then added an extraction step using a hybrid SpaCy2and Textacy3approach to extract the relevant triplets and construct the KG, illustrated in Figure 1. Since their approach works in an unsupervised end-to-end manner, it was vulnerable to inconsistency, as the output KG would depend on the structure of the statements in the corpus and the generated statements after the cloze querying to the language model. BertNet[24] tried to address this issue along with the dependency of having existing massive data to learn from. They proposed to apply a paraphrasing stage before extracting the triplets, that way, there would be a more diverse set of alternatives to generate the entities and the triplets from. And to handle the resulting issue of having a large search space after the paraphrasing, they proposed a search and scoring strategy to balance the accuracy and coverage of the output. Garg et al.[25] decided to go in a different direction and examine whether or not language models could capture graph semantics and if language models and graphs could work interchangeably. The objective was if a language model took a graph as an input can it output the same graph while maintaining the same semantics? They pointed out that due to the nature of language models, they take their input as a form of distributed representations or vectors, and to pass the graph as an input to a language model; it needs 1https://code.google.com/archive/p/relation-extraction-corpus/ 2SpaCy is a free, open-source library for NLP in Python. It’s written in Cython and is designed to build information extraction or natural language understanding systems 3Textacy is a Python library for performing a variety of natural language processing (NLP) tasks, built on the high-performance spaCy library. With the fundamentals — tokenization, part-of-speech tagging, dependency parsing, etc.Computer Science & Information Technology (CS & IT) 107Fig. 1. Pipeline to generate knowledge graphs from language models from [23] to be compressed into a vector representation which would affect its structural semantics. From this point, the experiments showed that transformer models could not express the full semantics of the input knowledge graph. Language models have shown so much potential when it comes to NLP tasks, and they are viewed as more than just processing models but potentially a pre-trained knowledge base. However, they store the knowledge in a form of a black box which still makes it difficult to access or edit. AlKhamissi et al.[26] provided a full analysis of language models and whether or not they could be considered a walking database. 2.3 Incremental Learning The aim of machine learning is to allow the system to behave as a human, and humans keep updating their knowledge either by trial and error, which is represented in the training phase, or by external knowledge and asking for the right answer, which is represented in the incremental learning phase of our approach. Incremental learning aims to keep the system updated and gives it the ability to learn new knowledge without having to retrain it on all the previous knowledge. There are three main ways to apply incremental learning; the first one is to have an interactive environment in which the system can get feedback and change its behavior accordingly, the second approach is to allow the system to get its knowledge from an external source which in our scope is shown with the human/expert in the loop, and the third is using both approaches together. The concept of incremental learning has been adapted to different domains, and fields [3,4,5,6]. Wang, Weikang, et al. followed the idea of building a training dataset consisting of five sub-datasets (SubD1, SubD2, SubD3, SubD4, and SubD5). The model is first trained on these sub-datasets one at a time till it is ready for deployment. But instead of using a traditional task-oriented dialogue system, they proposed a very interesting approach which is the Incremental Dialogue System (IDS)[27]. IDS uses the concept of adding a human-in-the-loop; the motivation of this approach is to reduce the non-relevant responses of the dialogue system. One of the main issues in dialogue systems is the irrelevant replies; since the model is usually trained on certain dialogues, it usually remembers ”most” of the replies, but in many cases, if the user asks a new question that the system didn’t see before then the response would be irrelevant. To fix that, first, they calculate the confidence level between the model’s reply and the user’s query; if it’s high, then respond with the generated system’s reply, but if it has a low confidence level meaning the reply is irrelevant, then let a human expert reply this time to the human. After asking the human in the loop to reply, the model needs to learn the108 Computer Science & Information Technology (CS & IT)answer to this user query, so they use incremental learning to do so, as shown in Figure 2. Their approach consists of mainly 3 modules: 1. Dialogue Embedding Module: at which the user utterance is embedded using Gated Recurrent Unit (GRU) based bidirectional Recurrent Neural Network (bi-RNN), and on top of it, they use self-attention layer to improve the encoding. 2. Uncertainty Estimation module: at which the confidence level between the user’s ut- terance and the system response(s) is calculated. 3. Online Learning module: this module is only used when the confidence level from the second module is low, and none of the candidate responses are relevant to the user utterance. In this case, a human expert is involved to respond on this utterance, and the system should be updated with the given utterance and its proper response using incremental learning. Fig. 2. Overview of IDS [27] The same idea of adding a human-in-the-loop was discussed with a different approach by Rajendran et al. [28]. Their goal was to maximize task success in task-oriented dialogue systems while minimizing the involvement of a human expert in the loop. This is very similar to the previous approach except that here they used reinforcement learning instead of incremental learning. The system has mainly three elements: the model M, a neural dialogue model which is trained for goal-oriented dialogues. The classifier C, which is a neural classifier that uses reinforcement learning to learn, and the human H, which is the expert in the loop. As discussed earlier, since it’s very unlikely to get users’ utterances to be similar to the ones in the training dataset, therefore the model’s response might not be correct. The idea is that with every user utterance, the classifier gets to pick between the model and the human; this way, the classifier learns with trial and error, using a reward and punishment system. If the classifier picked the model and the model answered correctly, the classifier gets rewarded with a high reward, but if the model answered incorrectly, then the punishment is high. Finally, if the classifier picked the human to answer, in this case, we always assume that the human response is correct, then the classifier is rewarded with a low reward. With trial and error, the classifier will try to maximize its reward by increasing the task success and minimizing the need for human.Computer Science & Information Technology (CS & IT) 109Our incremental learning approach is mostly inspired by [27,28], both of them worked with the motivation of improving the neural network task-oriented dialogue system by adding a human in the loop to respond whenever the system outputs an invalid/incorrect answer to the user, and takes this expert’s response and feeds it back to the system so it would learn the correct answer. 3 Proposed Method Our approach consists mainly of four modules; Knowledge Graph Extractor Module ,Knowl- edge Graph Embedding Module ,Incremental Learning Module , and Selection Module . The framework takes place over two phases, the first phase is the training phase, in which we construct the KG and train the model with the first two modules, and the second phase is the deployment phase. First, we construct the KG from a question-answer dataset for the training phase by taking each question and answer and converting them into statements. Then we pass these statements to OpenIE[29], which extracts the facts/triplets from each statement. And finally, we use the constructed KG to train our model. For the deployment phase, there are two scenarios; the first is that new knowledge needs to be added, so we use the Incremental Learning Module directly. The second is when interacting with the user; the user would ask a question, which we pass to the first module to convert into a statement to get the missing entity ( (h,r,?) tail prediction or (?,r,t) head prediction), then we pass this required prediction to the model to retrieve the missing entity. Here we have the following cases; –if the model could predict a link: •This link prediction is valid, so we output it to the user. •This link prediction is invalid, so we ask the human in the loop to answer and learn this new link. –if the model could not predict a link, which might happen if the source entity is not in the entities list that the model is trained on (new entity). In that case, we redirect this task to the human in the loop and feed the model the new fact to update it. 3.1 Knowledge Graph Extractor (KGE) Module This module aims to extract the KG facts from a Question-Answer dataset. Figure 3 illustrates an overview of this module. Having a question Qand answer A, we first pass Qto lexicalized PCFG parser[30] to extract the parse tree of the grammatical structure of the question. Fig. 3. Extracting Facts from QA dataset Generating a parse tree transforms a natural language sentence/question into its equiv- alent syntactic tree form representing the grammatical structure. This process includes identifying groups of words (phrases), part of speech tags of these phases/words, and de- pendency labels. This is done in an unsupervised manner by using a pre-trained parser110 Computer Science & Information Technology (CS & IT)provided by the Stanford Natural Language Processing Group4, which follows the English Penn Treebank5. Example is shown in Figure 4. Fig. 4. Output parse tree for ”To whom did the Virgin Mary allegedly appear in 1858 in Lourdes France?” After generating the parse tree, we use it to rewrite the question in the form of sentence S; by looping over the tree nodes. First, we decide where to place the given node in the sentence according to the node type. Then we remove the question header, and according to the type of question, we put the correct prepositions, if needed, before the answer. This sentence Swould contain a **blank** where the answer should be, so we simply take Aand replace it with **blank** . We use the same module for the user’s question to extract the fact with the missing entity to pass to the model for the link prediction while leaving the **blank** placeholder to identify the prediction needed. Once we have the sentence ready, we pass it to OpenIE[29] to extract all the possible fact triplets from the given sentence. The Open Information Extraction (OpenIE) is an unsupervised annotator that extracts the relation triples from a given system by splitting them into clauses. Each clause is used to generate a set of shorter sentence fragments. From these shorter sentences, it’s easier to extract the triples [31,32,29,33]. One of the main advantages of OpenIE is that the extracted facts are humanly readable, which makes it easier to generate the answer from the knowledge graph. The result of this module should be a knowledge graph KG={(e1,r,e2) ∥e1,e2∈Eand r ∈R}which contains the information presented in the question-answer pairs in the original dataset, which refers to the first contribution in the Introduction 1. 3.2 Knowledge Graph Embedding Module We used TransE as our KG embedding model for our experiments. TransE constructs the embeddings for the entities Ein∥E∥k, where k is the dimension of the embeddings which is passed as a hyperparameter to the model, and same for the relations Rin∥R∥k. Initially, these embeddings are randomly initialized, and the model gets to learn and fine-tune these 4https://nlp.stanford.edu/software/lex-parser.html 5http://surdeanu.cs.arizona.edu//mihai/teaching/ista555-fall13/readings/PennTreebankConstituents.htmlComputer Science & Information Technology (CS & IT) 111embeddings by minimizing the margin-based loss equation 2: L=X (e1,r,e2)∈SX (e1′,r,e2′)∈S′ (e1,r,e2)[γ+d(e1 +r, e2)−d(e1′+r, e2′)]+ (2) Where γ > 0 is a margin hyperparameter and the energy of a triplet d(e1′+r, e2′) is for some dissimilarity measured. The [ γ+d(e1 +r, e2)−d(e1′+r, e2′)]+donates only the resulted values that are positive. S′ (e1+r,e2)is the set of corrupted triplets, where it takes triplets from the training set and replaces either the head or tail in the triplet by a random entity but not both at the same time. Shown in equation 3: S′ (e1+r,e2)={(e1′, r, e2)|e1′∈E} ∪ { (e1, r, e2′)|e2′∈E}. (3) Because TransE is an energy-based model that uses a geometric interpretation of the latent space, it considers that a fact (e1,r,e2) holds when the embedding of the tail entity e2is close to the sum of the embedding of the head entity e1plus some vector that depends on the relationship r, i.e., e1+r≈e2. However, due to the nature of translation, TransE cannot correctly predict one-to-many and many-to-one relations; a selection mechanism is needed after the TransE prediction to help reduce this gap, further explained in module 3.4. 3.3 Incremental Learning Module After training the model, we use this module in two scenarios; 1) if there is new information that should be added to the knowledge base, or 2) when the user asks the system a question and the system provides an invalid answer. An overview of this module is illustrated in Figure 5. In the second scenario, the system can not provide a correct answer; hence, we redirect to the human expert to answer this question. According to that answer, we also use it to update the model and KG. Fig. 5. Incremental Learning with HITL112 Computer Science & Information Technology (CS & IT)As illustrated in Algorithm 1, first, we check if the ’new’ fact contains any new entities. In case it includes an entity that is not in the entity list of the KG, we first add it to the list of entities by adding a new dimension to the model’s entity parameter. Then, this new dimension is randomly initialized and updated by finetuning the model; by feeding it the new fact. The need for this step comes from the formal definition of the KG itself since the embeddings of the KG are dependent on the total number of entities and relations at the construction stage, as explained in subsection 3.2. If the new fact does not contain any new entity, then we move directly to feeding the model the ’new’6fact. This way, the model could keep learning new knowledge without affecting the previously learned knowledge and without retraining from the beginning, which refers to the second contribution in the Introduction Section1. Algorithm 1 Incremental Learning with the HITL feeding the model (e1,r,e2) 1:ife1/∈Ethen 2: E=E∪ {e1} 3: embed (e1) = random (size=k) ▷Randomly initializing embedding for the new entity 4: embed (E) =append (embed (E), embed (e1)) ▷Add randomly initialized vector for the new embedding in the embeddings 5:ife2/∈Ethen 6: E=E∪e2 7: embed (e2) = random (size=k) 8: embed (E) =append (embed (E), embed (e2)) 9: train for (e1,r,e2) 3.4 Selection Module This module is responsible for verifying the answer and selecting the correct answer output by the model since most KG embedding models mispredict the 1:n and n:1 relations, which makes sense because, without any provided context, all the facts are ’valid’. We can address this issue by passing the user’s question to the KGE module 3.1 and excluding the common facts between the questions and the valid facts predicted by the model, explained in Algorithm 2. This step reduces the gap of having n:1 or 1:n relations, which refers to the third contribution. Algorithm 2 Selection Module Algorithm 1:Input: Question q 2: 3:tree=ParseTree (q) ▷Generate the ParseTree q 4:sentence q=treetosentence (tree) ▷Generate the sentence of q without the answer 5:facts =OpenIE (sentence q) ▷Extract the facts and the missing fact to predict 6:missing fact =facts.contain 7:topkpredictions =model.predict (missing fact) ▷save top k predictions with the highest score 8:output fact =topkpredictions −facts ▷ Eliminate the common facts between the two sets 9: 10:Output: output fact ▷ Output the fact with the highest score 6The reason we put new between ” is that the fact that is being fed to the model might not be a new fact, but a fact that the model mispredict.Computer Science & Information Technology (CS & IT) 113This way, we have fewer predicted facts; if only one remains, the system outputs that to the user and waits for the user’s feedback. If it is not the correct fact that the user is looking for, then the system moves to HITL with the incremental module 3.3. If there is more than one valid fact after the filtration, the model picks the fact with the highest score as the correct output. If it is not the correct answer, move on to the next one until the user finds what they want. Finally, if none of the predicted facts were valid or no facts were left after the filtration, then we move to the HITL with the incremental module 3.3. For example: Q: To whom did the Virgin Mary allegedly appear in 1858 in Lourdes France? –from Qwe form the statement S: S: The Virgin Mary allegedly appeared in 1858 in Lourdes France to **blank** –from which we extract the following triplets: (The Virgin Mary, appear, in 1858) (The Virgin Mary, appear, in Lourdes France) (The Virgin Mary, appear, ?) –Then we pass this tail prediction to the model to get the missing entity, which would give us all the relevant facts; technically, all of them are correct/valid. (The Virgin Mary, appear, in 1858) (The Virgin Mary, appear, in Lourdes France) (The Virgin Mary, appear, Saint Bernadette Soubirous) –Since we know that the first two facts are already mentioned in the question, this is our context to pick the right answer; hence the system would output (The Virgin Mary, appear, Saint Bernadette Soubirous) which is the correct answer. 4 Experiments and Results In this section, we first start by introducing the datasets that we used in subsection 4.1, then we present the evaluation metrics that we used in subsection 4.2, and finally, we discuss our experiments and their results on the given datasets in subsection 4.3. 4.1 Datasets To validate our approach, we used the SQuAD[21] dataset, which is considered one of the benchmark datasets for the Question-Answering tasks. The SQuAD is a collection of question-answer pairs created from Wikipedia articles by humans through crowd-sourcing, which makes it very diverse. We used 70,000 question-answer pairs for our approach. After running the parser and the sentence generation step, we ended up with 68,445 sentences, which resulted in 100500 entities and 14783 relations, and 71194 facts after running the KGE module7. We used the cross-validation function by sklearn8to split our dataset into training, validation, and testing, each containing 56955 facts, 7119 facts, and 7120 facts, respectively. For the unseen experiments, we saved 50 entities away from the model among the corresponding 58 facts from the training, validation, and testing sets. We compared our approach with other approaches using language models from the literature review to construct the KG. We used the same framework from the methodology to convert the question-answer pairs into sentences. Still, instead of adding the answer to the sentence, they added ’[MASK]’ for the language model to learn how to fill it with the correct answer. We added this step to achieve the same task: constructing the KG 7We will refer to the KG dataset extracted from SQuAD with SQuADKG. 8https://scikit-learn.org/stable/114 Computer Science & Information Technology (CS & IT)from question-answer pairs, which is the main focus of our paper. We used the approach presented by[23] for this experiment with Roberta with their custom SpaCy and Textacy method to clean the KG. Each approach has its pros and cons, which we consider a designer choice for this step. Table 1 shows the results for running both methods on 57,355 statements as the parser and sentence generator output. On average, the LM approach took 03:33:52 to process 10,000 sentences, using GPUs offered by Colab Pro at the time we conducted this experiment which was Tesla T4, while for OpenIE, we used average processing power; Intel(R) Core i7-8750H CPU. Despite the difference in the computation power, the LM approach took a significantly longer time to process all the sentences and still extracted significantly fewer facts than the OpenIE approach; details about the LM experiment are discussed in Appendix section5.1. Using The LM approach would provide a cleaner graph in terms of entities and rela- tions. On the other hand, it requires multiple hops to reach the answer, which would affect the generation of the answer, as we have to keep track of the path taken to generate the answer. Although using OpenIE would make it easier to generate the answer, we might find facts like (The National Archives, make strides towards making its holdings more widely available In 2006) . These facts make it harder for the embedding model to understand the features and have many relations and entities with the same semantic meaning. However, this issue could be fixed by adding another layer of filtering the entities and relations before adding them to the graph. The last point of comparison is the execution time and resource needed for each, which is presented in the last row in Table 1; the time difference between the two methods is noticeable. POC LM OpenIE count entities 6,778 87,498 count relations 5,174 14,118 count facts 12,251 59,636 Execution time hh:mm:ss +20:00:00 02:10:27 Table 1. Constructing the KG with different approaches We continued the rest of the experiments with the dataset generated by OpenIE, as we wanted to reduce having implicit data learned by the language model and work with a more classical approach. In addition, we don’t see any improvement done by the LM, so we go with OpenIE as it requires less time and resources and still extracts more facts/knowledge. To validate the Incremental learning module on its own, we used YAGO3-10[34] and FB15K237[35] since these are the datasets that were included in the OpenKE[36] results. The YAGO3-10[34] consists of 123182 entities and 37 relations; it contains 1,079,040 facts for the training set and 5000 facts for each validation and testing set. We picked this dataset because its entities are associated with at least ten different relations. These relations describe human attributes like association, profession, gender, etc. The FB15K237[35] consists of 14,541 entities and 237 relations, which form 149678 facts for training, 3992 facts for testing, and 543 facts for validation. To simulate the incremental learning with human-in-the-loop, we removed 100entities from each of the datasets to save them for the unseen dataset along with their relevant facts from the training, validation and testing sets. These facts are then used to simulate the unseen queries for the model and apply the incremental learning approach. In addition to those, we saved any mistakenly predicted link while testing the model after trainingComputer Science & Information Technology (CS & IT) 115so that we get to add these incorrect facts to the testing set. For YAGO3-10, the total number of unseen facts was 1935 and 4992 mispredicted facts. For FB15K237, the total number of unseen facts was 77, and 20319 mispredicted facts. These three datasets are different in size, i.e., the total number of entities, relations, and the number of 1:n or n:1 relations. Since SQuADKG was constructed end-to-end, the number of relations is significantly larger than the number of relations of the other two datasets. 4.2 Metric Mean Rank (MR) : It is the average of the obtained ranks. Its range is between 1 and ∥E∥. As the value gets closer to 1 than ∥E∥, this means that the performance is improved. Because this metric is very sensitive to outliers, it’s usually not used just by itself. Mean Reciprocal Rank (MRR) : It’s value ranges between 0 and 1. It is the average of the inverse of the obtained ranks, and hence the higher the value is, the better the model results. Hits@K (H@K) : It is the ratio of predictions for which the rank is equal or lesser than a threshold K; its range lies between 0 and 1, where closer to 1 is better. Common values for K are 1, 3, 5, and 10. The higher the H@K, the better the model results. We mainly focus on K=10, which shows the proportion of correct entities ranked in the top 10. 4.3 Results and Analysis In Table 2, the TransE row shows the results of training TransE on the YAGO3-10 dataset. After saving some entities and facts for the incremental learning part, the model was trained for 500 epochs. The results of running the model on the given test file were 56.4% for hits@10. We saved the mispredicted facts to apply the incremental learning part to the model; this experiment aimed to simulate the human in the loop when the model mispredicted a particular fact, so we saved the facts that we knew that the model needed to ’learn.’ As expected, the results after this type of incremental learning improved the model’s performance on these facts. The results are shown in Table 2 in the TransE+ row. For the main experiment to feed the model entities that it has not seen before, we applied the incremental learning approach to this setting while adding a new dimension for this entity and initially randomizing its values till the model learns it. Then we went back to test the model’s performance on the unseen facts, as shown in Table 2 in the TransE++ unseen entities row that the model could recognize these entities and facts correctly after this learning. For the final experiment, we wanted to test if adding these new entities would affect the model’s performance on the original data of the training phase. So we merged all the facts from the three experiments (the original test set, the mispredicted facts, and the unseen facts). Row TransE++ all in Table 2 shows the results of this experiment that learning new entities does not affect TransE’s ability to recognize the original trained entities and facts. We repeated those four experiments with the same order for the FB15K237 dataset, and their results are shown in Table 3. TransE refers to training the model with the training dataset after removing the unseen facts and showing the results of running on the test set. TransE+ refers to the model after the incremental learning on the mispredicted facts, which was tested on those mispredicted facts after the incremental learning. One of the reasons why the performance did not reach high results for YAGO3-10 is the number116 Computer Science & Information Technology (CS & IT)Model MRR MR hit@10 hit@3 hit@1 TransE 0.342691 1704.469727 0.564413 0.402080 0.226445 TransE+ 0.895744 2.000801 0.988391 0.960268 0.828263 TransE++ unseen entities 0.788238 9.513178 0.928165 0.848579 0.707494 TransE++ all 0.796323 16.453110 0.940918 0.887318 0.697302 Table 2. Results of TransE on YAGO3-10. of 1:n and n:1 facts. TransE++ refers to the model after incrementally learning the unseen entities and their relevant facts. Here, we show the results of testing the model on just the unseen data, shown in row TransE++ unseen entities , and testing the model on all the data, shown in row TransE++ all . Model MRR MR hit@10 hit@3 hit@1 TransE 0.279723 233.155823 0.466185 0.317886 0.184267 TransE+ 0.489348 125.041397 0.622388 0.527722 0.412699 TransE++ unseen entities 0.744715 10.077922 0.831169 0.805195 0.681818 TransE++ all 0.452894 243.779419 0.576355 0.487343 0.382188 Table 3. Results of TransE on FB15K237 for the four scenarios. The drop-off in the performance of the model after training the model on new entities in some cases, for instance, in Table 3 when it dropped from 62% ( TransE+ ) to 57% (TransE++ all ), is acceptable in our approach because initially, the model cannot predict a link of an entity it has not been trained on, but now it has this capacity for these new entities. The objective of our approach is to allow the model to learn new facts and new entities while still being able to recognize the original facts with acceptable accuracy. So this is a trade-off between being unable to output a fact if it contains an entity it hasn’t seen and slightly affecting the prediction of existing facts. After constructing its knowledge graph entities and facts, we conducted the same experiments on the SQuADKG dataset. Results are shown in Table 4. The first scenario TransE was to train TransE while keeping the unseen entities and their corresponding facts outside the training set, the loss while training was 0.007913. We tested the trained model on the test set, which it had not seen before, not during the training or validation. The accuracy for hit@10 was almost 2.9%, which was perfect to showcase the effect of our incremental learning effect. The model could not predict the test facts correctly because we constructed the KG manually, which means that the graph would be vulnerable to being unconnected, especially after removing some for the unseen scenario and some for the test and validation sets. Model MRR MR hit@10 hit@3 hit@1 TransE 0.015392 42327.921875 0.029234 0.017569 0.007309 TransE+ 0.559917 62.463951 0.911174 0.865777 0.245678 TransE++ unseen entities 0.893739 1.844828 0.982759 0.956897 0.844828 TransE++ all 0.562405 62.002300 0.911683 0.865886 0.250244 Table 4. Results of TransE on SQuADKG for the four scenarios.Computer Science & Information Technology (CS & IT) 117The second scenario TransE+ was to teach the model these mispredicted/new facts and test it on the same set; here, we notice that the model was able to recognize these facts while still recognizing the other facts correctly as the score of the metrics got better, as the accuracy improved to 91% for these new facts (but not new entities). And finally, in the TransE++ where we teach the model entities and facts they have not seen before during the training phase, the prediction score when examining the model on the unseen facts was 98%, and to make sure that learning these new entities still did not affect its overall performance we tested the model on a merged test set that combines everything, at which it is shown that the model was still able to predict the facts with 91%. When using ComplEx on SQuADKG, we noticed that it performed poorly with an accuracy of less than 1% (0.009979) on the test set with a loss of 8.81. This could be due to the fact that the KG is constructed in an end-to-end unsupervised manner which highlights the problem with incompleteness, in addition to removing over 20% of the facts for validation and training, which makes the data significantly incomplete. Another reason is that ComplEx requires the relation r between the entities to be diagonal, as discussed in [7], which is not guaranteed in our case. Although TransE initially did not perform well either on this set, the difference in the scoring function and embeddings for each of the models made TransE more convenient to approach. 5 Conclusion and Future Work In this paper, we start by introducing our problem statement, which consists of two parts; the first is constructing a knowledge graph from a question-answering dataset since QA datasets are more available and easier to collect, unlike KGs. However, QA representations lack the structural representation that exists in data representations such as KGs. Another advantage of the KG representation is its ability to explicitly store the data, unlike deep language models, where data is stored implicitly and appears as a black box, making it harder to manipulate. For this problem, we propose our Knowledge Graph Extraction Module 3.1, which takes the question and extracts its grammatical structure using a parse tree generator. Then, from the generated tree, we rearrange the question in sentence form and add the answer in the correct position in the sentence. And the last step is to use OpenIE to extract the fact triples from the generated sentence. For the experiments, we used question-answer pairs from the SQuAD[21] dataset; from 70,000 question-answer pairs, we extracted 100500 entities and 14783 relations, and 71194 facts. The second part of our problem statement addresses the incompleteness problem, which is present in most of the knowledge representations but mainly highlighted in knowledge graphs because it is shown as missing links in the graph, making it harder for the model to reach the correct answer. For this problem, we propose an incremental learning approach that allows the model to learn from a human expert in the loop that would feed the model new knowledge when needed. This Incremental learning module also allows the model to learn new entities, not in the original training entity vocab list. The need for this step is shown when the user asks a link prediction query with an unseen entity or when the human-in-the-loop tries to feed the model a new fact that contains an unseen entity; in both cases, the model would typically raise an error. We conducted the same experiments on three different datasets to verify our frame- work. The first dataset is the extracted KG from the SQuAD dataset, which we refer to as SQuADKG. The second dataset is the YAGO3-10[34] dataset which is one of the bench- mark datasets that contains 123,182 entities, 37 relations, and 1,179,040 triples, and the118 Computer Science & Information Technology (CS & IT)third dataset is the FB15K237[35] which has 310,079 triples with 14,505 entities and 237 relations. Finally, for the embedding model, we used TransE[10]. We removed 100 entities with their related facts from the training, testing, and validation set to create the unseen set for the incremental learning experiments. For YAGO3-10, after training the results on the test dataset, TransE’s performance was 56.44%. Then, after the incremental learning on the mispredicted facts, it reached 98.83%. Finally, for the last experiment on the unseen entities facts and testing on the original test set, the performance was 94.09%. For FB15K237, the results were different due to the difference in the structure of the dataset itself. The experiments results on the test dataset were: initially 46.61%, after the incremental learning on the mispredicted facts, it reached 62.23%, and for the unseen entities facts with the Incremental Learning module, it achieved 83.11% on the unseen data. But on the complete test set, including the unseen data, the performance was 57.63%. The dropout from 62% to 57% is acceptable in our approach as it’s a trade-off between not having the needed vocab and mispredicting some facts since the incremental learning approach can handle this issue when required. Lastly, the SQuADKG dataset shows the most significant results, going from 2.92% in the first experiment to 91.16% in the final experiment on the unseen data. This im- provement makes sense since we constructed the KG and then removed over 20% of the dataset for the test, validation, and unseen data, leaving the training dataset significantly incomplete. That’s why the new knowledge fills the missing gaps in the constructed graph. Our final contribution was to address the 1:n and n:1 link prediction problem that is known to be one of TransE’s limitations, along with other models. We propose a selection method that eliminates the facts mentioned in the question to reduce the number of possible answers and supposedly output the response with the highest score. The objective was to try and reduce the complexity of each module, aiming to save resources while maintaining acceptable results. As for the limitations of our approach that we aim to target in our future work, start- ing with the resources and time efficiency, we found that the parser takes the most time. Although we tested this module on limited resources, we still aim to find a more efficient way to achieve the same results with a somewhat optimized approach. Regarding the embedding model, we want to experiment with different models and verify the effect of incremental learning, with the required tuning, on each of them. The incremental learn- ing approach would show promising results with reinforcement learning models because they naturally learn by interacting with the environment. One of the models that we are currently experimenting with is the dual agent approach[37] to navigate the KG. Finally, we aim to expand our approach to handle multi-turn conversations by adding another layer that would keep track of the current dialogue state and use it to filter the possible answers. That would help navigate the KG better without significantly increasing the complexity. References 1. Zheng Zhang, Ryuichi Takanobu, Qi Zhu, MinLie Huang, and XiaoYan Zhu. Recent advances and challenges in task-oriented dialog systems. Science China Technological Sciences , 63(10):2011–2027, 2020. 2. Xiaojun Chen, Shengbin Jia, and Yang Xiang. A review: Knowledge reasoning over knowledge graph. Expert Systems with Applications , 141:112948, 2020. 3. Samuel Budd, Emma C. Robinson, and Bernhard Kainz. A survey on active learning and human-in- the-loop deep learning for medical image analysis. Medical Image Analysis , 71:102062, 2021. 4. Andreas Holzinger, Markus Plass, Katharina Holzinger, Gloria Cerasela Cri¸ san, Camelia-M. Pintea, and Vasile Palade. Towards interactive machine learning (iml): Applying ant colony algorithms toComputer Science & Information Technology (CS & IT) 119solve the traveling salesman problem with the human-in-the-loop approach. In Francesco Buccafurri, Andreas Holzinger, Peter Kieseberg, A Min Tjoa, and Edgar Weippl, editors, Availability, Reliability, and Security in Information Systems , pages 81–95, Cham, 2016. Springer International Publishing. 5. Yiwei Yang, Eser Kandogan, Yunyao Li, Prithviraj Sen, and Walter S Lasecki. A study on interaction in human-in-the-loop machine learning for text analytics. In IUI Workshops , 2019. 6. Jiwei Li, Alexander H Miller, Sumit Chopra, Marc’Aurelio Ranzato, and Jason Weston. Dialogue learning with human-in-the-loop. arXiv preprint arXiv:1611.09823 , 2016. 7. Andrea Rossi, Denilson Barbosa, Donatella Firmani, Antonio Matinata, and Paolo Merialdo. Knowl- edge graph embedding for link prediction: A comparative analysis. ACM Transactions on Knowledge Discovery from Data (TKDD) , 15(2):1–49, 2021. 8. Th´ eo Trouillon, Johannes Welbl, Sebastian Riedel, ´Eric Gaussier, and Guillaume Bouchard. Complex embeddings for simple link prediction. In International conference on machine learning , pages 2071– 2080. PMLR, 2016. 9. Maximilian Nickel, Lorenzo Rosasco, and Tomaso Poggio. Holographic embeddings of knowledge graphs. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 30, 2016. 10. Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. Advances in neural information processing systems , 26, 2013. 11. Wen Zhang, Bibek Paudel, Wei Zhang, Abraham Bernstein, and Huajun Chen. Interaction embeddings for prediction and explanation in knowledge graphs. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining , pages 96–104, 2019. 12. Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. Rotate: Knowledge graph embedding by relational rotation in complex space. arXiv preprint arXiv:1902.10197 , 2019. 13. Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2d knowl- edge graph embeddings. In Proceedings of the AAAI conference on artificial intelligence , volume 32, 2018. 14. Dai Quoc Nguyen, Tu Dinh Nguyen, Dat Quoc Nguyen, and Dinh Phung. A novel embedding model for knowledge base completion based on convolutional neural network. arXiv preprint arXiv:1712.02121 , 2017. 15. Jill Burstein, Christy Doran, and Thamar Solorio. Proceedings of the 2019 conference of the north american chapter of the association for computational linguistics: Human language technologies, vol- ume 1 (long and short papers). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , 2019. 16. Thanh Vu, Tu Dinh Nguyen, Dat Quoc Nguyen, Dinh Phung, et al. A capsule network-based embed- ding model for knowledge graph completion and search personalization. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pages 2180–2189, 2019. 17. Lingbing Guo, Zequn Sun, and Wei Hu. Learning to exploit long-term relational dependencies in knowledge graphs. In International Conference on Machine Learning , pages 2505–2514. PMLR, 2019. 18. A. H. Miller P. Lewis A. Bakhtin Y. Wu F. Petroni, T. Rockt¨ aschel and S. Riedel. Language models as knowledge bases? In In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2019 , 2019. 19. Hady Elsahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon Hare, Frederique Laforest, and Elena Simperl. T-rex: A large scale alignment of natural language with knowledge base triples. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018) , 2018. 20. Robyn Speer, Catherine Havasi, et al. Representing general relational knowledge in conceptnet 5. In LREC , volume 2012, pages 3679–86, 2012. 21. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 , 2016. 22. Chenguang Wang, Xiao Liu, and Dawn Song. Language models are open knowledge graphs. arXiv preprint arXiv:2010.11967 , 2020. 23. Vinitra Swamy, Angelika Romanou, and Martin Jaggi. Interpreting language models through knowl- edge graph extraction. arXiv preprint arXiv:2111.08546 , 2021. 24. Shibo Hao, Bowen Tan, Kaiwen Tang, Hengzhe Zhang, Eric P Xing, and Zhiting Hu. Bertnet: Har- vesting knowledge graphs from pretrained language models. arXiv preprint arXiv:2206.14268 , 2022. 25. Tarun Garg, Kaushik Roy, and Amit Sheth. Can language models capture graph semantics? from graphs to language model and vice-versa. arXiv preprint arXiv:2206.09259 , 2022. 26. Badr AlKhamissi, Millicent Li, Asli Celikyilmaz, Mona Diab, and Marjan Ghazvininejad. A review on language models as knowledge bases. arXiv preprint arXiv:2204.06031 , 2022.120 Computer Science & Information Technology (CS & IT)27. Weikang Wang, Jiajun Zhang, Qian Li, Mei-Yuh Hwang, Chengqing Zong, and Zhifei Li. Incremental learning from scratch for task-oriented dialogue systems. arXiv preprint arXiv:1906.04991 , 2019. 28. Janarthanan Rajendran, Jatin Ganhotra, and Lazaros C. Polymenakos. Learning End-to-End Goal- Oriented Dialog with Maximal User Task Success and Minimal Human Agent Use. Transactions of the Association for Computational Linguistics , 7:375–386, 07 2019. 29. Mausam Mausam. Open information extraction systems and downstream applications. In Proceedings of the twenty-fifth international joint conference on artificial intelligence , pages 4074–4077, 2016. 30. Dan Klein and Christopher D Manning. Accurate unlexicalized parsing. In Proceedings of the 41st annual meeting of the association for computational linguistics , pages 423–430, 2003. 31. Swarnadeep Saha et al. Open information extraction from conjunctive sentences. In Proceedings of the 27th International Conference on Computational Linguistics , pages 2288–2299, 2018. 32. Swarnadeep Saha, Harinder Pal, et al. Bootstrapping for numerical open ie. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages 317–323, 2017. 33. Harinder Pal et al. Demonyms and compound relational nouns in nominal open ie. In Proceedings of the 5th Workshop on Automated Knowledge Base Construction , pages 35–39, 2016. 34. Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2d knowl- edge graph embeddings. In Proceedings of the AAAI conference on artificial intelligence , volume 32, 2018. 35. Kristina Toutanova and Danqi Chen. Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd workshop on continuous vector space models and their composi- tionality , pages 57–66, 2015. 36. Xu Han, Shulin Cao, Lv Xin, Yankai Lin, Zhiyuan Liu, Maosong Sun, and Juanzi Li. Openke: An open toolkit for knowledge embedding. In Proceedings of EMNLP , 2018. 37. Denghui Zhang, Zixuan Yuan, Hao Liu, Xiaodong Lin, and Hui Xiong. Learning to walk with dual agents for knowledge graph reasoning. arXiv preprint arXiv:2112.12876 , 2021. Appendix 5.1 Using Language Model to Generate KG In this section, we discuss the experiment of constructing the KG using a language model (LM). The objective of this experiment was to compare and explore other approaches for constructing the KG. Starting with running the experiment on the approach proposed in [23] for which they proposed using masked sentences and passing them to the LM to allow the model to learn the features of the given sentence and accordingly extract the fact triplets. Then they filter those facts using a hybrid layer of SpaCy and Textacy to verify the extracted facts. In Table 5, we present the results of each phase for this experiment. The GTcolumn refers to the Ground Truth facts extracted using the SpaCy and Textacy libraries, the LM column refers to the facts predicted by the language model, the Missed GT column refers to the facts missed by the LM and captured in the ground truth. The New LM refers to the facts captured by the LM but not by the ground truth, and finally, the Intersection column is the facts captured by both the LM and the ground truth. From observing the results of each phase, by manually evaluating samples of each, we noticed that the relevant facts are the intersection between the LM and the ground truth. However, the number of extracted facts was too little compared to the time and resources taken to extract them. When experimenting with the SpaCy and Textacy layer to extract the facts to see if this filtering step could improve the quality of the facts extracted from OpenIE, we took aComputer Science & Information Technology (CS & IT) 121POC GT LM Missed GT New LM Intersection count entities 11,666 6,778 9,593 4,077 4,161 count relations 6,901 5,174 4,907 2,682 3,085 count facts 14,885 12,251 11,330 7,470 5,878 Table 5. Extracting Facts Using Language Model sample of 10,000 sentences from the question-answer processed pairs. The execution time was 1 hour, 30 minutes, and 27 seconds to generate 3048 facts that contain 3060 entities and 1840 relations. This filtering step could be an extra cleaning and verification step. Still, within our project, we didn’t notice any significant improvement after applying this step relative to the time taken to process the whole dataset. Authors Mervat Abu-Elkheir Associate Professor and vice-dean at German University in Cairo. She has over 20 years of experience in academic teaching. Her current research interests are in the areas of interpretable machine learning and AI, AI for software engineering, data management, and engineering, and natural language understanding. She has been an IEEE member since 2014. Mayar Osama received a Bachelor of Science in Media Engineering and Technology from German University in Cairo. Currently, she is pursuing her Master of Computer Science while working as a teaching assistant. Her research interests include Natural Language Processing, and Knowledge Reasoning and Representation122 Computer Science & Information Technology (CS & IT) © 2023 By AIRCC Publishing Corporation. This article is published under the Creative Commons Attribution (CC BY) license.

Paper 5a8dd8b644049fdfc877cb15e1c4a9b249ea1c5c not found in buffer: ['410e4e70ec38f9a4cfd43df6dd92adc9f739ce2a', '40e3cd767c9... Paper 5a8dd8b644049fdfc877cb15e1c4a9b249ea1c5c not found in buffer: ['410e4e70ec38f9a4cfd43df6dd92adc9f739ce2a', '40e3cd767c9520171ac8ecbd2e27f3a1bf043b61', 'd777d15677b673fe02750333181ea5e1e6dadf17', 'fe351bc1bda19583a7d6b1a96a9b18f59c40b213', '441068ef17c3332d4f0af6c146e37bd8748bb90e', '781402fae3a7a44b90855eeba8e96a872347fd4b', '3a6933886485038805e7af23a38845f9d93e1b9c', 'c70cefca5c76724b27e4615259be889d9ca3aa92', '42d83576ee920c1b6df318e212047d9ba57fc4fd', '7fff3df60b8d9a5e57627f4eb6b3022b60f3a39d']. Please try a different paper. Paper 5a8dd8b644049fdfc877cb15e1c4a9b249ea1c5c not found in buffer: ['410e4e70ec38f9a4cfd43df6dd92adc9f739ce2a', '40e3cd767c9520171ac8ecbd2e27f3a1bf043b61', 'd777d15677b673fe02750333181ea5e1e6dadf17', 'fe351bc1bda19583a7d6b1a96a9b18f59c40b213', '441068ef17c3332d4f0af6c146e37bd8748bb90e', '781402fae3a7a44b90855eeba8e96a872347fd4b', '3a6933886485038805e7af23a38845f9d93e1b9c', 'c70cefca5c76724b27e4615259be889d9ca3aa92', '42d83576ee920c1b6df318e212047d9ba57fc4fd', '7fff3df60b8d9a5e57627f4eb6b3022b60f3a39d']. Please try a different paper.

Caution, you have reached the maximum number of actions. Please select a paper. Caution, you have reached the maximum number of actions. Please select a paper. Caution, you have reached the maximum number of actions. Please select a paper.