spb/wp44_uqo

1{"work_id": 2923943538, "title": "FIESTA : fast incremental euclidean distance fields for online motion planning of aerial robots", "abstract": "991012769067103412 HKUST Electronic Theses FIESTA : fast incremental euclidean distance fields for online motion planning of aerial robots by Luxin Han thesis 2019 x, 42 pages : illustrations ; 30 cm As the rapid development of robotics, Micro Aerial Vehicles\u2026Read more \u203a"}2{"work_id": 3169320628, "title": "HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units", "abstract": "Self-supervised approaches for speech representation learning are challenged by three unique problems: (1) there are multiple sound units in each input utterance, (2) there is no lexicon of input sound units during the pre-training phase, and (3) sound units have variable lengths with no explicit segmentation. To deal with these three problems, we propose the Hidden-Unit BERT (HuBERT) approach for self-supervised speech representation learning, which utilizes an offline clustering step to provide aligned target labels for a BERT-like prediction loss. A key ingredient of our approach is applying the prediction loss over the masked regions only, which forces the model to learn a combined acoustic and language model over the continuous inputs. HuBERT relies primarily on the consistency of the unsupervised clustering step rather than the intrinsic quality of the assigned cluster labels. Starting with a simple k-means teacher of 100 clusters, and using two iterations of clustering, the HuBERT model either matches or improves upon the state-of-the-art wav2vec 2.0 performance on the Librispeech (960h) and Libri-light (60,000h) benchmarks with 10min, 1h, 10h, 100h, and 960h fine-tuning subsets. Using a 1B parameter model, HuBERT shows up to 19% and 13% relative WER reduction on the more challenging dev-other and test-other evaluation subsets."}3{"work_id": 2995327724, "title": "Analyzing and Improving the Image Quality of StyleGAN", "abstract": "The style-based GAN architecture (StyleGAN) yields state-of-the-art results in data-driven unconditional generative image modeling. We expose and analyze several of its characteristic artifacts, and propose changes in both model architecture and training methods to address them. In particular, we redesign the generator normalization, revisit progressive growing, and regularize the generator to encourage good conditioning in the mapping from latent codes to images. In addition to improving image quality, this path length regularizer yields the additional benefit that the generator becomes significantly easier to invert. This makes it possible to reliably attribute a generated image to a particular network. We furthermore visualize how well the generator utilizes its output resolution, and identify a capacity problem, motivating us to train larger models for additional quality improvements. Overall, our improved model redefines the state of the art in unconditional image modeling, both in terms of existing distribution quality metrics as well as perceived image quality."}4{"work_id": 4385318467, "title": "3D Gaussian Splatting for Real-Time Radiance Field Rendering", "abstract": "Radiance Field methods have recently revolutionized novel-view synthesis of scenes captured with multiple photos or videos. However, achieving high visual quality still requires neural networks that are costly to train and render, while recent faster methods inevitably trade off speed for quality. For unbounded and complete scenes (rather than isolated objects) and 1080p resolution rendering, no current method can achieve real-time display rates. We introduce three key elements that allow us to achieve state-of-the-art visual quality while maintaining competitive training times and importantly allow high-quality real-time (\u2265 30 fps) novel-view synthesis at 1080p resolution. First, starting from sparse points produced during camera calibration, we represent the scene with 3D Gaussians that preserve desirable properties of continuous volumetric radiance fields for scene optimization while avoiding unnecessary computation in empty space; Second, we perform interleaved optimization/density control of the 3D Gaussians, notably optimizing anisotropic covariance to achieve an accurate representation of the scene; Third, we develop a fast visibility-aware rendering algorithm that supports anisotropic splatting and both accelerates training and allows realtime rendering. We demonstrate state-of-the-art visual quality and real-time rendering on several established datasets."}5{"work_id": 4221142221, "title": "BERTopic: Neural topic modeling with a class-based TF-IDF procedure", "abstract": "Topic models can be useful tools to discover latent topics in collections of documents. Recent studies have shown the feasibility of approach topic modeling as a clustering task. We present BERTopic, a topic model that extends this process by extracting coherent topic representation through the development of a class-based variation of TF-IDF. More specifically, BERTopic generates document embedding with pre-trained transformer-based language models, clusters these embeddings, and finally, generates topic representations with the class-based TF-IDF procedure. BERTopic generates coherent topics and remains competitive across a variety of benchmarks involving classical models and those that follow the more recent clustering approach of topic modeling."}6{"work_id": 3127751679, "title": "TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation", "abstract": "Medical image segmentation is an essential prerequisite for developing healthcare systems, especially for disease diagnosis and treatment planning. On various medical image segmentation tasks, the u-shaped architecture, also known as U-Net, has become the de-facto standard and achieved tremendous success. However, due to the intrinsic locality of convolution operations, U-Net generally demonstrates limitations in explicitly modeling long-range dependency. Transformers, designed for sequence-to-sequence prediction, have emerged as alternative architectures with innate global self-attention mechanisms, but can result in limited localization abilities due to insufficient low-level details. In this paper, we propose TransUNet, which merits both Transformers and U-Net, as a strong alternative for medical image segmentation. On one hand, the Transformer encodes tokenized image patches from a convolution neural network (CNN) feature map as the input sequence for extracting global contexts. On the other hand, the decoder upsamples the encoded features which are then combined with the high-resolution CNN feature maps to enable precise localization. We argue that Transformers can serve as strong encoders for medical image segmentation tasks, with the combination of U-Net to enhance finer details by recovering localized spatial information. TransUNet achieves superior performances to various competing methods on different medical applications including multi-organ segmentation and cardiac segmentation. Code and models are available at https://github.com/Beckschen/TransUNet."}7{"work_id": 3138516171, "title": "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows", "abstract": "This paper presents a new vision Transformer, called Swin Transformer, that capably serves as a general-purpose backbone for computer vision. Challenges in adapting Transformer from language to vision arise from differences between the two domains, such as large variations in the scale of visual entities and the high resolution of pixels in images compared to words in text. To address these differences, we propose a hierarchical Transformer whose representation is computed with Shifted windows. The shifted windowing scheme brings greater efficiency by limiting self-attention computation to non-overlapping local windows while also allowing for cross-window connection. This hierarchical architecture has the flexibility to model at various scales and has linear computational complexity with respect to image size. These qualities of Swin Transformer make it compatible with a broad range of vision tasks, including image classification (87.3 top-1 accuracy on ImageNet-1K) and dense prediction tasks such as object detection (58.7 box AP and 51.1 mask AP on COCO test-dev) and semantic segmentation (53.5 mIoU on ADE20K val). Its performance surpasses the previous state-of-the-art by a large margin of +2.7 box AP and +2.6 mask AP on COCO, and +3.2 mIoU on ADE20K, demonstrating the potential of Transformer-based models as vision backbones. The hierarchical design and the shifted window approach also prove beneficial for all-MLP architectures. The code and models are publicly available at https://github.com/microsoft/Swin-Transformer."}8{"work_id": 1506965148, "title": "Mastering the Information Age Solving Problems with Visual Analytics", "abstract": "Today, in many spheres of human activity, massive amounts of data are collected and stored. As the volumes of data available to lawmakers, civil servants, business people and scientists increase, their effective use becomes more challenging. Simply keeping up to date with the flood of data, using standard tools for data management and analysis, is fraught with difficulty. The field of visual analytics seeks to provide people with better and more effective means to understand and analyse massive amounts of data. It also seeks to enable them to act upon their findings immediately, that is, in real time. Visual analytics integrates the analytic capabilities of the computer and the abilities of the human analyst. In so doing it allows novel discoveries and empowers individuals to take control of the analytical process. Visual analytics opens the door to the discoveries and unexpected insights which lead to beneficial and profitable innovation. This book is the result of the VisMaster project, a coordination action funded by the European Commission to form a strong visual analytics research community."}9{"work_id": 2963420686, "title": "Squeeze-and-Excitation Networks", "abstract": "The central building block of convolutional neural networks (CNNs) is the convolution operator, which enables networks to construct informative features by fusing both spatial and channel-wise information within local receptive fields at each layer. A broad range of prior research has investigated the spatial component of this relationship, seeking to strengthen the representational power of a CNN by enhancing the quality of spatial encodings throughout its feature hierarchy. In this work, we focus instead on the channel relationship and propose a novel architectural unit, which we term the \"Squeeze-and-Excitation\" (SE) block, that adaptively recalibrates channel-wise feature responses by explicitly modelling interdependencies between channels. We show that these blocks can be stacked together to form SENet architectures that generalise extremely effectively across different datasets. We further demonstrate that SE blocks bring significant improvements in performance for existing state-of-the-art CNNs at slight additional computational cost. Squeeze-and-Excitation Networks formed the foundation of our ILSVRC 2017 classification submission which won first place and reduced the top-5 error to 2.251 percent, surpassing the winning entry of 2016 by a relative improvement of \u223c 25 percent. Models and code are available at https://github.com/hujie-frank/SENet."}10{"work_id": 2982399380, "title": "BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension", "abstract": "Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, Luke Zettlemoyer. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020."}11{"work_id": 2963341956, "title": "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding", "abstract": "Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019."}12{"work_id": 3035524453, "title": "Momentum Contrast for Unsupervised Visual Representation Learning", "abstract": "We present Momentum Contrast (MoCo) for unsupervised visual representation learning. From a perspective on contrastive learning as dictionary look-up, we build a dynamic dictionary with a queue and a moving-averaged encoder. This enables building a large and consistent dictionary on-the-fly that facilitates contrastive unsupervised learning. MoCo provides competitive results under the common linear protocol on ImageNet classification. More importantly, the representations learned by MoCo transfer well to downstream tasks. MoCo can outperform its supervised pre-training counterpart in 7 detection/segmentation tasks on PASCAL VOC, COCO, and other datasets, sometimes surpassing it by large margins. This suggests that the gap between unsupervised and supervised representation learning has been largely closed in many vision tasks."}13{"work_id": 1524333225, "title": "Kaldi Speech Recognition Toolkit", "abstract": "Abstract\u2014We describe the design of Kaldi, a free, open-source toolkit for speech recognition research. Kaldi provides a speech recognition system based on finite-state transducers (using the freely available OpenFst), together with detailed documentation and scripts for building complete recognition systems. Kaldi is written is C++, and the core library supports modeling of arbitrary phonetic-context sizes, acoustic modeling with subspace Gaussian mixture models (SGMM) as well as standard Gaussian mixture models, together with all commonly used linear and affine transforms. Kaldi is released under the Apache License v2.0, which is highly nonrestrictive, making it suitable for a wide community of users. I."}14{"work_id": 2896457183, "title": "AI-Assisted Pipeline for Dynamic Generation of Trustworthy Health Supplement Content at Scale", "abstract": "Although geospatial question answering systems have received increasing attention in recent years, existing prototype systems struggle to properly answer qualitative spatial questions. In this work, we propose a unique framework for answering qualitative spatial questions, which comprises three main components: a geoparser that takes the input questions and extracts place semantic information from text, a reasoning system which is embedded with a crisp reasoner, and finally, answer extraction, which refines the solution space and generates final answers. We present an experimental design to evaluate our framework for point-based cardinal direction calculus (CDC) relations by developing an automated approach for generating three types of synthetic qualitative spatial questions. The initial evaluations of generated answers in our system are promising because a high proportion of answers were labelled correct."}15{"work_id": 3042011474, "title": "CSPNet: A New Backbone that can Enhance Learning Capability of CNN", "abstract": "Neural networks have enabled state-of-the-art approaches to achieve incredible results on computer vision tasks such as object detection. However, such success greatly relies on costly computation resources, which hinders people with cheap devices from appreciating the advanced technology. In this paper, we propose Cross Stage Partial Network (CSPNet) to mitigate the problem that previous works require heavy inference computations from the network architecture perspective. We attribute the problem to the duplicate gradient information within network optimization. The proposed networks respect the variability of the gradients by integrating feature maps from the beginning and the end of a network stage, which, in our experiments, reduces computations by 20% with equivalent or even superior accuracy on the ImageNet dataset, and significantly outperforms state-of-the-art approaches in terms of AP50on the MS COCO object detection dataset. The CSPNet is easy to implement and general enough to cope with architectures based on ResNet, ResNeXt, and DenseNet."}16{"work_id": 2900368655, "title": "SDR \u2013 Half-baked or Well Done?", "abstract": "In speech enhancement and source separation, signal-to-noise ratio is a ubiquitous objective measure of denoising/separation quality. A decade ago, the BSS_eval toolkit was developed to give researchers worldwide a way to evaluate the quality of their algorithms in a simple, fair, and hopefully insightful way: it attempted to account for channel variations, and to not only evaluate the total distortion in the estimated signal but also split it in terms of various factors such as remaining interference, newly added artifacts, and channel errors. In recent years, hundreds of papers have been relying on this toolkit to evaluate their proposed methods and compare them to previous works, often arguing that differences on the order of 0.1 dB proved the effectiveness of a method over others. We argue here that the signal-to-distortion ratio (SDR) implemented in the BSS_eval toolkit has generally been improperly used and abused, especially in the case of single-channel separation, resulting in misleading results. We propose to use a slightly modified definition, resulting in a simpler, more robust measure, called scale-invariant SDR (SI-SDR). We present various examples of critical failure of the original SDR that SI-SDR overcomes."}17{"work_id": 2911489562, "title": "BioBERT: a pre-trained biomedical language representation model for biomedical text mining", "abstract": "MOTIVATION: Biomedical text mining is becoming increasingly important as the number of biomedical documents rapidly grows. With the progress in natural language processing (NLP), extracting valuable information from biomedical literature has gained popularity among researchers, and deep learning has boosted the development of effective biomedical text mining models. However, directly applying the advancements in NLP to biomedical text mining often yields unsatisfactory results due to a word distribution shift from general domain corpora to biomedical corpora. In this article, we investigate how the recently introduced pre-trained language model BERT can be adapted for biomedical corpora. RESULTS: We introduce BioBERT (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining), which is a domain-specific language representation model pre-trained on large-scale biomedical corpora. With almost the same architecture across tasks, BioBERT largely outperforms BERT and previous state-of-the-art models in a variety of biomedical text mining tasks when pre-trained on biomedical corpora. While BERT obtains performance comparable to that of previous state-of-the-art models, BioBERT significantly outperforms them on the following three representative biomedical text mining tasks: biomedical named entity recognition (0.62% F1 score improvement), biomedical relation extraction (2.80% F1 score improvement) and biomedical question answering (12.24% MRR improvement). Our analysis results show that pre-training BERT on biomedical corpora helps it to understand complex biomedical texts. AVAILABILITY AND IMPLEMENTATION: We make the pre-trained weights of BioBERT freely available at https://github.com/naver/biobert-pretrained, and the source code for fine-tuning BioBERT available at https://github.com/dmis-lab/biobert."}18{"work_id": 3016007010, "title": "State of the Art on Neural Rendering", "abstract": "Abstract Efficient rendering of photo\u2010realistic virtual worlds is a long standing effort of computer graphics. Modern graphics techniques have succeeded in synthesizing photo\u2010realistic images from hand\u2010crafted scene representations. However, the automatic generation of shape, materials, lighting, and other aspects of scenes remains a challenging problem that, if solved, would make photo\u2010realistic computer graphics more widely accessible. Concurrently, progress in computer vision and machine learning have given rise to a new approach to image synthesis and editing, namely deep generative models. Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e.g., by the integration of differentiable rendering into network training. With a plethora of applications in computer graphics and vision, neural rendering is poised to become a new area in the graphics community, yet no survey of this emerging field exists. This state\u2010of\u2010the\u2010art report summarizes the recent trends and applications of neural rendering. We focus on approaches that combine classic computer graphics techniques with deep generative models to obtain controllable and photorealistic outputs. Starting with an overview of the underlying computer graphics and machine learning concepts, we discuss critical aspects of neural rendering approaches. Specifically, our emphasis is on the type of control, i.e., how the control is provided, which parts of the pipeline are learned, explicit vs. implicit control, generalization, and stochastic vs. deterministic synthesis. The second half of this state\u2010of\u2010the\u2010art report is focused on the many important use cases for the described algorithms such as novel view synthesis, semantic photo manipulation, facial and body reenactment, relighting, free\u2010viewpoint video, and the creation of photo\u2010realistic avatars for virtual and augmented reality telepresence. Finally, we conclude with a discussion of the social implications of such technology and investigate open research problems."}19{"work_id": 3177052299, "title": "Coordinate Attention for Efficient Mobile Network Design", "abstract": "Recent studies on mobile network design have demonstrated the remarkable effectiveness of channel attention (e.g., the Squeeze-and-Excitation attention) for lifting model performance, but they generally neglect the positional information, which is important for generating spatially selective attention maps. In this paper, we propose a novel attention mechanism for mobile networks by embedding positional information into channel attention, which we call \"coordinate attention\". Unlike channel attention that transforms a feature tensor to a single feature vector via 2D global pooling, the coordinate attention factorizes channel attention into two 1D feature encoding processes that aggregate features along the two spatial directions, respectively. In this way, long-range dependencies can be captured along one spatial direction and meanwhile precise positional information can be preserved along the other spatial direction. The resulting feature maps are then encoded separately into a pair of direction-aware and position-sensitive attention maps that can be complementarily applied to the input feature map to augment the representations of the objects of interest. Our coordinate attention is simple and can be flexibly plugged into classic mobile networks, such as MobileNetV2, MobileNeXt, and EfficientNet with nearly no computational overhead. Extensive experiments demonstrate that our coordinate attention is not only beneficial to ImageNet classification but more interestingly, behaves better in down-stream tasks, such as object detection and semantic segmentation. Code is available at https://github.com/Andrew-Qibin/CoordAttention."}20{"work_id": 4402128637, "title": "The promise and challenges of generative AI in education", "abstract": "Generative artificial intelligence (GenAI) tools, such as large language models (LLMs), generate natural language and other types of content to perform a wide range of tasks. This represents a significant technological advancement that poses opportunities and challenges to educational research and practice. This commentary brings together contributions from nine experts working in the intersection of learning and technology and presents critical reflections on the opportunities, challenges, and implications related to GenAI technologies in the context of education. In the commentary, it is acknowledged that GenAI\u2019s capabilities can enhance some teaching and learning practices, such as learning design, regulation of learning, automated content, feedback, and assessment. Nevertheless, we also highlight its limitations, potential disruptions, ethical consequences, and potential misuses. The identified avenues for further research include the development of new insights into the roles human experts can play, strong and continuous evidence, human-centric design of technology, necessary policy, and support and competence mechanisms. Overall, we concur with the general skeptical optimism about the use of GenAI tools such as LLMs in education. Moreover, we highlight the danger of hastily adopting GenAI tools in education without deep consideration of the efficacy, ecosystem-level implications, ethics, and pedagogical soundness of such practices."}21{"work_id": 4392233542, "title": "A Review of Machine Learning Approaches for the Personalization of Amplification in Hearing Aids", "abstract": "This paper provides a review of various machine learning approaches that have appeared in the literature aimed at individualizing or personalizing the amplification settings of hearing aids. After stating the limitations associated with the current one-size-fits-all settings of hearing aid prescriptions, a spectrum of studies in engineering and hearing science are discussed. These studies involve making adjustments to prescriptive values in order to enable preferred and individualized settings for a hearing aid user in an audio environment of interest to that user. This review gathers, in one place, a comprehensive collection of works that have been conducted thus far with respect to achieving the personalization or individualization of the amplification function of hearing aids. Furthermore, it underscores the impact that machine learning can have on enabling an improved and personalized hearing experience for hearing aid users. This paper concludes by stating the challenges and future research directions in this area."}22{"work_id": 2998508940, "title": "Random Erasing Data Augmentation", "abstract": "In this paper, we introduce Random Erasing, a new data augmentation method for training the convolutional neural network (CNN). In training, Random Erasing randomly selects a rectangle region in an image and erases its pixels with random values. In this process, training images with various levels of occlusion are generated, which reduces the risk of over-fitting and makes the model robust to occlusion. Random Erasing is parameter learning free, easy to implement, and can be integrated with most of the CNN-based recognition models. Albeit simple, Random Erasing is complementary to commonly used data augmentation techniques such as random cropping and flipping, and yields consistent improvement over strong baselines in image classification, object detection and person re-identification. Code is available at: https://github.com/zhunzhong07/Random-Erasing."}23{"work_id": 2952218014, "title": "Conv-TasNet: Surpassing Ideal Time\u2013Frequency Magnitude Masking for Speech Separation", "abstract": "Single-channel, speaker-independent speech separation methods have recently seen great progress. However, the accuracy, latency, and computational cost of such methods remain insufficient. The majority of the previous methods have formulated the separation problem through the time-frequency representation of the mixed signal, which has several drawbacks, including the decoupling of the phase and magnitude of the signal, the suboptimality of time-frequency representation for speech separation, and the long latency of the entire system. To address these shortcomings, we propose a fully-convolutional time-domain audio separation network (Conv-TasNet), a deep learning framework for end-to-end time-domain speech separation. Conv-TasNet uses a linear encoder to generate a representation of the speech waveform optimized for separating individual speakers. Speaker separation is achieved by applying a set of weighting functions (masks) to the encoder output. The modified encoder representations are then inverted back to the waveforms using a linear decoder. The masks are found using a temporal convolutional network (TCN) consisting of stacked 1-D dilated convolutional blocks, which allows the network to model the long-term dependencies of the speech signal while maintaining a small model size. The proposed Conv-TasNet system significantly outperforms previous time-frequency masking methods in separating two- and three-speaker mixtures. Additionally, Conv-TasNet surpasses several ideal time-frequency magnitude masks in two-speaker speech separation as evaluated by both objective distortion measures and subjective quality assessment by human listeners. Finally, Conv-TasNet has a significantly smaller model size and a much shorter minimum latency, making it a suitable solution for both offline and real-time speech separation applications. This study therefore represents a major step toward the realization of speech separation systems for real-world speech processing technologies."}24{"work_id": 2784874046, "title": "ArcFace: Additive Angular Margin Loss for Deep Face Recognition", "abstract": "Recently, a popular line of research in face recognition is adopting margins in the well-established softmax loss function to maximize class separability. In this paper, we first introduce an Additive Angular Margin Loss (ArcFace), which not only has a clear geometric interpretation but also significantly enhances the discriminative power. Since ArcFace is susceptible to the massive label noise, we further propose sub-center ArcFace, in which each class contains K sub-centers and training samples only need to be close to any of the K positive sub-centers. Sub-center ArcFace encourages one dominant sub-class that contains the majority of clean faces and non-dominant sub-classes that include hard or noisy faces. Based on this self-propelled isolation, we boost the performance through automatically purifying raw web faces under massive real-world noise. Besides discriminative feature embedding, we also explore the inverse problem, mapping feature vectors to face images. Without training any additional generator or discriminator, the pre-trained ArcFace model can generate identity-preserved face images for both subjects inside and outside the training data only by using the network gradient and Batch Normalization (BN) priors. Extensive experiments demonstrate that ArcFace can enhance the discriminative feature embedding as well as strengthen the generative face synthesis."}25{"work_id": 2983040767, "title": "Unsupervised Cross-lingual Representation Learning at Scale", "abstract": "Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzm\u00e1n, Edouard Grave, Myle Ott, Luke Zettlemoyer, Veselin Stoyanov. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020."}26{"work_id": 4318351475, "title": "MuChoMusic dataset", "abstract": "MuChoMusic: Evaluating Music Understanding in Multimodal Audio-Language Models MuChoMusic is a benchmark designed to evaluate music understanding in multimodal language models focused on audio. It includes 1,187 multiple-choice questions validated by human annotators, based on 644 music tracks from two publicly available music datasets. These questions cover a wide variety of genres and assess knowledge and reasoning across several musical concepts and their cultural and functional contexts. The benchmark provides a holistic evaluation of five open-source models, revealing challenges such as over-reliance on the language modality and highlighting the need for better multimodal integration. Note on Audio Files This dataset comes without audio files. The audio files can be downloaded from two datasets: SongDescriberDataset (SDD) and MusicCaps. Please see the code repository for more information on how to download the audio. Citation If you use this dataset, please cite our paper: @inproceedings{weck2024muchomusic, title={MuChoMusic: Evaluating Music Understanding in Multimodal Audio-Language Models}, author={Weck, Benno and Manco, Ilaria and Benetos, Emmanouil and Quinton, Elio and Fazekas, Gy\u00f6rgy and Bogdanov, Dmitry}, booktitle = {Proceedings of the 25th International Society for Music Information Retrieval Conference (ISMIR)}, year={2024} }"}27{"work_id": 4404534210, "title": "A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions", "abstract": "The emergence of large language models (LLMs) has marked a significant breakthrough in natural language processing (NLP), fueling a paradigm shift in information acquisition. Nevertheless, LLMs are prone to hallucination, generating plausible yet nonfactual content. This phenomenon raises significant concerns over the reliability of LLMs in real-world information retrieval (IR) systems and has attracted intensive research to detect and mitigate such hallucinations. Given the open-ended general-purpose attributes inherent to LLMs, LLM hallucinations present distinct challenges that diverge from prior task-specific models. This divergence highlights the urgency for a nuanced understanding and comprehensive overview of recent advances in LLM hallucinations. In this survey, we begin with an innovative taxonomy of hallucination in the era of LLM and then delve into the factors contributing to hallucinations. Subsequently, we present a thorough overview of hallucination detection methods and benchmarks. Our discussion then transfers to representative methodologies for mitigating LLM hallucinations. Additionally, we delve into the current limitations faced by retrieval-augmented LLMs in combating hallucinations, offering insights for developing more robust IR systems. Finally, we highlight the promising research directions on LLM hallucinations, including hallucination in large vision-language models and understanding of knowledge boundaries in LLM hallucinations."}28{"work_id": 4360884927, "title": "Diffusion Models in Vision: A Survey", "abstract": "Denoising diffusion models represent a recent emerging topic in computer vision, demonstrating remarkable results in the area of generative modeling. A diffusion model is a deep generative model that is based on two stages, a forward diffusion stage and a reverse diffusion stage. In the forward diffusion stage, the input data is gradually perturbed over several steps by adding Gaussian noise. In the reverse stage, a model is tasked at recovering the original input data by learning to gradually reverse the diffusion process, step by step. Diffusion models are widely appreciated for the quality and diversity of the generated samples, despite their known computational burdens, i.e., low speeds due to the high number of steps involved during sampling. In this survey, we provide a comprehensive review of articles on denoising diffusion models applied in vision, comprising both theoretical and practical contributions in the field. First, we identify and present three generic diffusion modeling frameworks, which are based on denoising diffusion probabilistic models, noise conditioned score networks, and stochastic differential equations. We further discuss the relations between diffusion models and other deep generative models, including variational auto-encoders, generative adversarial networks, energy-based models, autoregressive models and normalizing flows. Then, we introduce a multi-perspective categorization of diffusion models applied in computer vision. Finally, we illustrate the current limitations of diffusion models and envision some interesting directions for future research."}29{"work_id": 7152514513, "title": "Cooperative 4D trajectory planning for multi aerial vehicles combining receding horizon optimization and differential evolution algorithm in dynamic battlefield environment", "abstract": "Aiming to the problem of cooperative trajectory planning for multi aerial vehicles in dynamic battlefield environment, this article develops a cooperative four-dimensional trajectory planning method combining receding horizon optimization (RHO) and differential evolution algorithm, and the main contributions are threefold: (1) An effective framework for multi aerial vehicles cooperative trajectory planning in dynamic environment is introduced. Therein, RHO serves as the outer layer, while the DE algorithm is employed in the inner loop to implement local trajectory planning. (2) A population diversity-based adaptive DE algorithm, named PDADE, is proposed to tackle the issues of slow convergence speed and low convergence accuracy in the classical DE algorithm. (3) A flight speed adjustment strategy based on the required time of arrival is presented to ensure synchronization of arrival times. Eventually, some numerical simulations are conducted, and the performance of the proposed PDADE algorithm is compared with other seven state-of-the-art swarm optimization algorithms. The results demonstrate that PDADE achieves optimal convergence accuracy and convergence speed."}30{"work_id": 4366548330, "title": "Why Johnny Can\u2019t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts", "abstract": "Pre-trained large language models (\u201cLLMs\u201d) like GPT-3 can engage in fluent, multi-turn instruction-taking out-of-the-box, making them attractive materials for designing natural language interactions. Using natural language to steer LLM outputs (\u201cprompting\u201d) has emerged as an important design technique potentially accessible to non-AI-experts. Crafting effective prompts can be challenging, however, and prompt-based interactions are brittle. Here, we explore whether non-AI-experts can successfully engage in \u201cend-user prompt engineering\u201d using a design probe\u2014a prototype LLM-based chatbot design tool supporting development and systematic evaluation of prompting strategies. Ultimately, our probe participants explored prompt designs opportunistically, not systematically, and struggled in ways echoing end-user programming systems and interactive machine learning systems. Expectations stemming from human-to-human instructional experiences, and a tendency to overgeneralize, were barriers to effective prompt design. These findings have implications for non-AI-expert-facing LLM-based tool design and for improving LLM-and-prompt literacy among programmers and the public, and present opportunities for further research."}31{"work_id": 4383097638, "title": "Code as Policies: Language Model Programs for Embodied Control", "abstract": "Large language models (LLMs) trained on code-completion have been shown to be capable of synthesizing simple Python programs from docstrings [1]. We find that these code-writing LLMs can be re-purposed to write robot policy code, given natural language commands. Specifically, policy code can express functions or feedback loops that process perception outputs (e.g., from object detectors [2], [3]) and parameterize control primitive APIs. When provided as input several example language commands (formatted as comments) followed by corresponding policy code (via few-shot prompting), LLMs can take in new commands and autonomously re-compose API calls to generate new policy code respectively. By chaining classic logic structures and referencing third-party libraries (e.g., NumPy, Shapely) to perform arithmetic, LLMs used in this way can write robot policies that (i) exhibit spatial-geometric reasoning, (ii) generalize to new instructions, and (iii) prescribe precise values (e.g., velocities) to ambiguous descriptions (\u2018faster\u2019) depending on context (i.e., behavioral commonsense). This paper presents Code as Policies: a robot-centric formulation of language model generated programs (LMPs) that can represent reactive policies (e.g., impedance controllers), as well as waypoint-based policies (vision-based pick and place, trajectory-based control), demonstrated across multiple real robot platforms. Central to our approach is prompting hierarchical code-gen (recursively defining undefined functions), which can write more complex code and also improves state-of-the-art to solve 39.8% of problems on the HumanEval [1] benchmark. Code and videos are available at https://code-as-policies.github.io"}32{"work_id": 4404301029, "title": "The Metaverse Review: Exploring the Boundless Ream of Digital Reality", "abstract": "The metaverse has emerged as a prominent topic with growing interest fueled by advancements in Web 3.0, blockchain, and immersive technologies. This paper presents a thorough analysis of the metaverse, showcasing its evolutio... | Find, read and cite all the research you need on Tech Science Press"}33{"work_id": 2936774411, "title": "SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition", "abstract": "We present SpecAugment, a simple data augmentation method for speech recognition. SpecAugment is applied directly to the feature inputs of a neural network (i.e., filter bank coefficients). The augmentation policy consists of warping the features, masking blocks of frequency channels, and masking blocks of time steps. We apply SpecAugment on Listen, Attend and Spell networks for end-to-end speech recognition tasks. We achieve state-of-the-art performance on the LibriSpeech 960h and Swichboard 300h tasks, outperforming all prior work. On LibriSpeech, we achieve 6.8% WER on test-other without the use of a language model, and 5.8% WER with shallow fusion with a language model. This compares to the previous state-of-the-art hybrid system of 7.5% WER. For Switchboard, we achieve 7.2%/14.6% on the Switchboard/CallHome portion of the Hub5'00 test set without the use of a language model, and 6.8%/14.1% with shallow fusion, which compares to the previous state-of-the-art hybrid system at 8.3%/17.3% WER."}34{"work_id": 4385430679, "title": "RT-1: Robotics Transformer for Real-World Control at Scale", "abstract": "By transferring knowledge from large, diverse, taskagnostic datasets, modern machine learning models can solve specific downstream tasks either zero-shot or with small taskspecific datasets to a high level of performance.While this capability has been demonstrated in other fields such as computer vision, natural language processing or speech recognition, it remains to be shown in robotics, where the generalization capabilities of the models are particularly critical due to the difficulty of collecting real-world robotic data.We argue that one of the keys to the success of such general robotic models lies with open-ended task-agnostic training, combined with highcapacity architectures that can absorb all of the diverse, robotic data.In this paper, we present a model class, dubbed Robotics Transformer, that exhibits promising scalable model properties.We verify our conclusions in a study of different model classes and their ability to generalize as a function of the data size, model size, and data diversity based on a large-scale data collection on real robots performing real-world tasks.How would you bring me two sodas?How would you move three cokes to the trash can?How would you throw away two cokes?How would you bring me two different sodas?How would you bring me an apple, a coke, and water bottle?I spilled my coke on the table, how would you throw it away and then bring me something to help clean?I just worked out, can you bring me a drink and a snack to recover?How would you bring me a fruit, a soda, and a bag of chips for lunchTABLE XI: List of SayCan instructions evaluated in Sec.IV-D"}35{"work_id": 3207138404, "title": "Metaverse for Social Good", "abstract": "In recent years, the metaverse has attracted enormous attention from around the world with the development of related technologies. The expected metaverse should be a realistic society with more direct and physical interactions, while the concepts of race, gender, and even physical disability would be weakened, which would be highly beneficial for society. However, the development of metaverse is still in its infancy, with great potential for improvement. Regarding metaverse's huge potential, industry has already come forward with advance preparation, accompanied by feverish investment, but there are few discussions about metaverse in academia to scientifically guide its development. In this paper, we highlight the representative applications for social good. Then we propose a three-layer metaverse architecture from a macro perspective, containing infrastructure, interaction, and ecosystem. Moreover, we journey toward both a historical and novel metaverse with a detailed timeline and table of specific attributes. Lastly, we illustrate our implemented blockchain-driven metaverse prototype of a university campus and discuss the prototype design and insights."}36{"work_id": 4210598935, "title": "A Review of Yolo Algorithm Developments", "abstract": "Object detection techniques are the foundation for the artificial intelligence field. This research paper gives a brief overview of the You Only Look Once (YOLO) algorithm and its subsequent advanced versions. Through the analysis, we reach many remarks and insightful results. The results show the differences and similarities among the YOLO versions and between YOLO and Convolutional Neural Networks (CNNs). The central insight is the YOLO algorithm improvement is still ongoing.This article briefly describes the development process of the YOLO algorithm, summarizes the methods of target recognition and feature selection, and provides literature support for the targeted picture news and feature extraction in the financial and other fields. Besides, this paper contributes a lot to YOLO and other object detection literature."}37{"work_id": 3184439416, "title": "YOLOX: Exceeding YOLO Series in 2021", "abstract": "In this report, we present some experienced improvements to YOLO series, forming a new high-performance detector -- YOLOX. We switch the YOLO detector to an anchor-free manner and conduct other advanced detection techniques, i.e., a decoupled head and the leading label assignment strategy SimOTA to achieve state-of-the-art results across a large scale range of models: For YOLO-Nano with only 0.91M parameters and 1.08G FLOPs, we get 25.3% AP on COCO, surpassing NanoDet by 1.8% AP; for YOLOv3, one of the most widely used detectors in industry, we boost it to 47.3% AP on COCO, outperforming the current best practice by 3.0% AP; for YOLOX-L with roughly the same amount of parameters as YOLOv4-CSP, YOLOv5-L, we achieve 50.0% AP on COCO at a speed of 68.9 FPS on Tesla V100, exceeding YOLOv5-L by 1.8% AP. Further, we won the 1st Place on Streaming Perception Challenge (Workshop on Autonomous Driving at CVPR 2021) using a single YOLOX-L model. We hope this report can provide useful experience for developers and researchers in practical scenes, and we also provide deploy versions with ONNX, TensorRT, NCNN, and Openvino supported. Source code is at https://github.com/Megvii-BaseDetection/YOLOX."}38{"work_id": 4391136507, "title": "A Survey on Evaluation of Large Language Models", "abstract": "Large language models (LLMs) are gaining increasing popularity in both academia and industry, owing to their unprecedented performance in various applications. As LLMs continue to play a vital role in both research and daily use, their evaluation becomes increasingly critical, not only at the task level, but also at the society level for better understanding of their potential risks. Over the past years, significant efforts have been made to examine LLMs from various perspectives. This paper presents a comprehensive review of these evaluation methods for LLMs, focusing on three key dimensions: what to evaluate , where to evaluate , and how to evaluate . Firstly, we provide an overview from the perspective of evaluation tasks, encompassing general natural language processing tasks, reasoning, medical usage, ethics, education, natural and social sciences, agent applications, and other areas. Secondly, we answer the \u2018where\u2019 and \u2018how\u2019 questions by diving into the evaluation methods and benchmarks, which serve as crucial components in assessing the performance of LLMs. Then, we summarize the success and failure cases of LLMs in different tasks. Finally, we shed light on several future challenges that lie ahead in LLMs evaluation. Our aim is to offer invaluable insights to researchers in the realm of LLMs evaluation, thereby aiding the development of more proficient LLMs. Our key point is that evaluation should be treated as an essential discipline to better assist the development of LLMs. We consistently maintain the related open-source materials at: https://github.com/MLGroupJLU/LLM-eval-survey"}39{"work_id": 4312933868, "title": "High-Resolution Image Synthesis with Latent Diffusion Models", "abstract": "By decomposing the image formation process into a sequential application of denoising autoencoders, diffusion models (DMs) achieve state-of-the-art synthesis results on image data and beyond. Additionally, their formulation allows for a guiding mechanism to control the image generation process without retraining. However, since these models typically operate directly in pixel space, optimization of powerful DMs often consumes hundreds of GPU days and inference is expensive due to sequential evaluations. To enable DM training on limited computational resources while retaining their quality and flexibility, we apply them in the latent space of powerful pretrained autoencoders. In contrast to previous work, training diffusion models on such a representation allows for the first time to reach a near-optimal point between complexity reduction and detail preservation, greatly boosting visual fidelity. By introducing cross-attention layers into the model architecture, we turn diffusion models into powerful and flexible generators for general conditioning inputs such as text or bounding boxes and high-resolution synthesis becomes possible in a convolutional manner. Our latent diffusion models (LDMs) achieve new state of the art scores for image inpainting and class-conditional image synthesis and highly competitive performance on various tasks, including unconditional image generation, text-to-image synthesis, and super-resolution, while significantly reducing computational requirements compared to pixel-based DMs."}40{"work_id": 7205708264, "title": "Relationship between educational robotics and executive functions in children with neurodevelopmental disorders", "abstract": "International audience"}41{"work_id": 4372260310, "title": "Large-Scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation", "abstract": "Contrastive learning has shown remarkable success in the field of multimodal representation learning. In this paper, we propose a pipeline of contrastive language-audio pretraining to develop an audio representation by combining audio data with natural language descriptions. To accomplish this target, we first release LAION-Audio-630K, a large collection of 633,526 audio-text pairs from different data sources. Second, we construct a contrastive language-audio pretraining model by considering different audio encoders and text encoders. We incorporate the feature fusion mechanism and keyword-to-caption augmentation into the model design to further enable the model to process audio inputs of variable lengths and enhance the performance. Third, we perform comprehensive experiments to evaluate our model across three tasks: text-to-audio retrieval, zero-shot audio classification, and supervised audio classification. The results demonstrate that our model achieves superior performance in text-to-audio retrieval task. In audio classification tasks, the model achieves state-of-the-art performance in the zero-shot setting and is able to obtain performance comparable to models\u2019 results in the non-zero-shot setting. LAION-Audio-630K1and the proposed model2are both available to the public."}42{"work_id": 4411403346, "title": "A Comprehensive Overview of Large Language Models", "abstract": "Large Language Models (LLMs) have recently demonstrated remarkable capabilities in natural language processing tasks and beyond. This success of LLMs has led to a large influx of research contributions in this direction. These works encompass diverse topics such as architectural innovations, better training strategies, context length improvements, fine-tuning, multimodal LLMs, robotics, datasets, benchmarking, efficiency, and more. With the rapid development of techniques and regular breakthroughs in LLM research, it has become considerably challenging to perceive the bigger picture of the advances in this direction. Considering the rapidly emerging plethora of literature on LLMs, it is imperative that the research community is able to benefit from a concise yet comprehensive overview of the recent developments in this field. This article provides an overview of the literature on a broad range of LLM-related concepts. Our self-contained comprehensive overview of LLMs discusses relevant background concepts along with covering the advanced topics at the frontier of research in LLMs. This review article is intended to provide not only a systematic survey but also a quick, comprehensive reference for the researchers and practitioners to draw insights from extensive, informative summaries of the existing works to advance the LLM research."}43{"work_id": 3212386989, "title": "Attention mechanisms in computer vision: A survey", "abstract": "Humans can naturally and effectively find salient regions in complex scenes. Motivated by this observation, attention mechanisms were introduced into computer vision with the aim of imitating this aspect of the human visual system. Such an attention mechanism can be regarded as a dynamic weight adjustment process based on features of the input image. Attention mechanisms have achieved great success in many visual tasks, including image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multimodal tasks, and self-supervised learning. In this survey, we provide a comprehensive review of various attention mechanisms in computer vision and categorize them according to approach, such as channel attention, spatial attention, temporal attention, and branch attention; a related repository https://github.com/MenghaoGuo/Awesome-Vision-Attentions is dedicated to collecting related work. We also suggest future directions for attention mechanism research."}44{"work_id": 4404826297, "title": "Prompting AI Art: An Investigation into the Creative Skill of Prompt Engineering", "abstract": "We are witnessing a novel era of creativity where anyone can create digital content via prompt-based learning (known as prompt engineering). This article investigates prompt engineering as a novel creative skill for creating AI art with text-to-image generation. In three consecutive studies, we explore whether crowdsourced participants can (1) discern prompt quality, (2) write prompts, and (3) refine prompts. We find that participants could evaluate prompt quality and crafted descriptive prompts, but they lacked style-specific vocabulary necessary for effective prompting. This is in line with our hypothesis that prompt engineering is a new type of skill that is non-intuitive and must first be acquired (e.g., through means of practice and learning) before it can be used at a level of high quality. Our studies deepen our understanding of prompt engineering and chart future research directions. We conclude by envisioning four potential futures for prompt engineering."}45{"work_id": 4391876619, "title": "Lost in the Middle: How Language Models Use Long Contexts", "abstract": "Abstract While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context. We analyze the performance of language models on two tasks that require identifying relevant information in their input contexts: multi-document question answering and key-value retrieval. We find that performance can degrade significantly when changing the position of relevant information, indicating that current language models do not robustly make use of information in long input contexts. In particular, we observe that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models. Our analysis provides a better understanding of how language models use their input context and provides new evaluation protocols for future long-context language models."}46{"work_id": 4386047745, "title": "Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks", "abstract": "To design fast neural networks, many works have been focusing on reducing the number of floating-point operations (FLOPs). We observe that such reduction in FLOPs, however, does not necessarily lead to a similar level of re-duction in latency. This mainly stems from inefficiently low floating-point operations per second (FLOPS). To achieve faster networks, we revisit popular operators and demonstrate that such low FLOPS is mainly due to frequent memory access of the operators, especially the depthwise con-volution. We hence propose a novel partial convolution (PConv) that extracts spatial features more efficiently, by cutting down redundant computation and memory access simultaneously. Building upon our PConv, we further propose FasterNet, a new family of neural networks, which attains substantially higher running speed than others on a wide range of devices, without compromising on accuracy for various vision tasks. For example, on ImageNet-lk, our tiny FasterNet-TO is 2.8\u00d7, 3.3\u00d7, and 2.4\u00d7 faster than MobileViT-XXS on GPU, CPU, and ARM processors, respectively, while being 2.9% more accurate. Our large FasterNet-L achieves impressive 83.5% top-1 accuracy, on par with the emerging Swin-B, while having 36% higher inference throughput on GPU, as well as saving 37% compute time on CPU. Code is available at https://github.com/JierunChen/FasterNet."}47{"work_id": 4226002507, "title": "Masked Autoencoders Are Scalable Vision Learners", "abstract": "This paper shows that masked autoencoders (MAE) are scalable self-supervised learners for computer vision. Our MAE approach is simple: we mask random patches of the input image and reconstruct the missing pixels. It is based on two core designs. First, we develop an asymmetric encoder-decoder architecture, with an encoder that operates only on the visible subset of patches (without mask tokens), along with a lightweight decoder that reconstructs the original image from the latent representation and mask tokens. Second, we find that masking a high proportion of the input image, e.g., 75%, yields a nontrivial and meaningful self-supervisory task. Coupling these two designs enables us to train large models efficiently and effectively: we accelerate training (by 3\u00d7 or more) and improve accuracy. Our scalable approach allows for learning high-capacity models that generalize well: e.g., a vanilla ViT-Huge model achieves the best accuracy (87.8%) among methods that use only ImageNet-1K data. Transfer performance in downstream tasks outperforms supervised pretraining and shows promising scaling behavior."}48{"work_id": 2787560479, "title": "Deep Contextualized Word Representations", "abstract": "Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, Luke Zettlemoyer. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018."}49{"work_id": 2954996726, "title": "A survey on Image Data Augmentation for Deep Learning", "abstract": "Deep convolutional neural networks have performed remarkably well on many Computer Vision tasks. However, these networks are heavily reliant on big data to avoid overfitting. Overfitting refers to the phenomenon when a network learns a function with very high variance such as to perfectly model the training data. Unfortunately, many application domains do not have access to big data, such as medical image analysis. This survey focuses on Data Augmentation, a data-space solution to the problem of limited data. Data Augmentation encompasses a suite of techniques that enhance the size and quality of training datasets such that better Deep Learning models can be built using them. The image augmentation algorithms discussed in this survey include geometric transformations, color space augmentations, kernel filters, mixing images, random erasing, feature space augmentation, adversarial training, generative adversarial networks, neural style transfer, and meta-learning. The application of augmentation methods based on GANs are heavily covered in this survey. In addition to augmentation techniques, this paper will briefly discuss other characteristics of Data Augmentation such as test-time augmentation, resolution impact, final dataset size, and curriculum learning. This survey will present existing methods for Data Augmentation, promising developments, and meta-level decisions for implementing Data Augmentation. Readers will understand how Data Augmentation can improve the performance of their models and expand limited datasets to take advantage of the capabilities of big data."}50{"work_id": 4390873054, "title": "Adding Conditional Control to Text-to-Image Diffusion Models", "abstract": "We present ControlNet, a neural network architecture to add spatial conditioning controls to large, pretrained text-to-image diffusion models. ControlNet locks the production-ready large diffusion models, and reuses their deep and robust encoding layers pretrained with billions of images as a strong backbone to learn a diverse set of conditional controls. The neural architecture is connected with \"zero convolutions\" (zero-initialized convolution layers) that progressively grow the parameters from zero and ensure that no harmful noise could affect the finetuning. We test various conditioning controls, e.g., edges, depth, segmentation, human pose, etc., with Stable Diffusion, using single or multiple conditions, with or without prompts. We show that the training of ControlNets is robust with small (1m) datasets. Extensive results show that ControlNet may facilitate wider applications to control image diffusion models."}51{"work_id": 4320002812, "title": "Object Detection in 20 Years: A Survey", "abstract": "Object detection, as of one the most fundamental and challenging problems in computer vision, has received great attention in recent years. Over the past two decades, we have seen a rapid technological evolution of object detection and its profound impact on the entire computer vision field. If we consider today\u2019s object detection technique as a revolution driven by deep learning, then, back in the 1990s, we would see the ingenious thinking and long-term perspective design of early computer vision. This article extensively reviews this fast-moving research field in the light of technical evolution, spanning over a quarter-century\u2019s time (from the 1990s to 2022). A number of topics have been covered in this article, including the milestone detectors in history, detection datasets, metrics, fundamental building blocks of the detection system, speedup techniques, and recent state-of-the-art detection methods."}52{"work_id": 2798812533, "title": "Universal Language Model Fine-tuning for Text Classification", "abstract": "Inductive transfer learning has greatly impacted computer vision, but existing approaches in NLP still require task-specific modifications and training from scratch.We propose Universal Language Model Fine-tuning (ULMFiT), an effective transfer learning method that can be applied to any task in NLP, and introduce techniques that are key for fine-tuning a language model.Our method significantly outperforms the state-of-the-art on six text classification tasks, reducing the error by 18-24% on the majority of datasets.Furthermore, with only 100 labeled examples, it matches the performance of training from scratch on 100\u00d7 more data.We opensource our pretrained models and code 1 ."}53{"work_id": 4372266552, "title": "CLAP Learning Audio Concepts from Natural Language Supervision", "abstract": "Mainstream machine listening models are trained to learn audio concepts under the paradigm of one class label to many recordings focusing on one task. Learning under such restricted supervision limits the flexibility of models because they require labeled audio for training and can only predict the predefined categories. Instead, we propose to learn audio concepts from natural language supervision. We call our approach Contrastive Language-Audio Pretraining (CLAP), which connects language and audio by using two encoders and a contrastive learning objective, bringing audio and text descriptions into a joint multimodal space. We trained CLAP with 128k audio and text pairs and evaluated it on 16 downstream tasks across 7 domains, such as classification of sound events, scenes, music, and speech. CLAP establishes state-of-the-art (SoTA) in Zero-Shot performance. Also, we evaluated CLAP\u2019s audio encoder in a supervised learning setup and achieved SoTA in 5 tasks. The Zero-Shot capability removes the need of training with class labeled audio, enables flexible class prediction at inference time, and generalizes well in multiple downstream tasks. Code is available at: https://github.com/microsoft/CLAP."}54{"work_id": 4385482704, "title": "Photo tourism: exploring photo collections in 3D", "abstract": "We present a system for interactively browsing and exploring large unstructured collections of photographs of a scene using a novel 3D interface. Our system consists of an image-based modeling front end that automatically computes the viewpoint of each photograph as well as a sparse 3D model of the scene and image to model correspondences. Our photo explorer uses image-based rendering techniques to smoothly transition between photographs, while also enabling full 3D navigation and exploration of the set of images and world geometry, along with auxiliary information such as overhead maps. Our system also makes it easy to construct photo tours of scenic or historic locations, and to annotate image details, which are automatically transferred to other relevant images. We demonstrate our system on several large personal photo collections as well as images gathered from Internet photo sharing sites."}55{"work_id": 2308543850, "title": "Scene-motion- and latency-perception thresholds for head-mounted displays", "abstract": "A fundamental task of an immersive virtual environment (IVE) system is to present images of the virtual world that change appropriately as the user's head moves. Current IVE systems, especially those using head-mounted displays (HMDs), often produce spatially unstable scenes, resulting in simulator sickness, degraded task performance, degraded visual acuity, and breaks in presence. In HMDs, instability resulting from latency is greater than all other causes of instability combined. The primary way users perceive latency in an HMD is by improper motion of scenes that should be stationary in the world. Whereas latency-induced scene motion is well defined mathematically, less is understood about how much scene motion and/or latency can occur without subjects noticing, and how this varies under different conditions. I built a simulated HMD system with zero effective latency---no scene motion occurs due to latency. I intentionally and artificially inserted scene motion into the virtual environment in order to determine how much scene motion and/or latency can occur without subjects noticing. I measured perceptual thresholds of scene-motion and latency under different conditions across five experiments. Based on the study of latency, head motion, scene motion, and perceptual thresholds, I developed a mathematical model of latency thresholds as an inverse function of peak head-yaw acceleration. Psychophysics studies showed that measured latency thresholds correlate with this inverse function better than with a linear function. The work reported here readily enables scientists and engineers to, under their particular conditions, measure latency thresholds as a function of head motion by using an off-the-shelf projector system. Latency requirements can thus be determined before designing HMD systems."}56{"work_id": 4386076325, "title": "YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors", "abstract": "Real-time object detection is one of the most important research topics in computer vision. As new approaches regarding architecture optimization and training optimization are continually being developed, we have found two research topics that have spawned when dealing with these latest state-of-the-art methods. To address the topics, we propose a trainable bag-of-freebies oriented solution. We combine the flexible and efficient training tools with the proposed architecture and the compound scaling method. YOLOv7 surpasses all known object detectors in both speed and accuracy in the range from 5 FPS to 120 FPS and has the highest accuracy 56.8% AP among all known real-time object detectors with 30 FPS or higher on GPU V100. Source code is released in https://github.com/WongKinYiu/yolov7."}57{"work_id": 4206484811, "title": "A Metaverse: Taxonomy, Components, Applications, and Open Challenges", "abstract": "Unlike previous studies on the Metaverse based on Second Life, the current Metaverse is based on the social value of Generation Z that online and offline selves are not different. With the technological development of deep learning-based high-precision recognition models and natural generation models, Metaverse is being strengthened with various factors, from mobile-based always-on access to connectivity with reality using virtual currency. The integration of enhanced social activities and neural-net methods requires a new definition of Metaverse suitable for the present, different from the previous Metaverse. This paper divides the concepts and essential techniques necessary for realizing the Metaverse into three components (i.e., hardware, software, and contents) and three approaches (i.e., user interaction, implementation, and application) rather than marketing or hardware approach to conduct a comprehensive analysis. Furthermore, we describe essential methods based on three components and techniques to Metaverse\u2019s representative Ready Player One, Roblox, and Facebook research in the domain of films, games, and studies. Finally, we summarize the limitations and directions for implementing the immersive Metaverse as social influences, constraints, and open challenges."}58{"work_id": 2884367402, "title": "Object Detection With Deep Learning: A Review", "abstract": "Due to object detection's close relationship with video analysis and image understanding, it has attracted much research attention in recent years. Traditional object detection methods are built on handcrafted features and shallow trainable architectures. Their performance easily stagnates by constructing complex ensembles that combine multiple low-level image features with high-level context from object detectors and scene classifiers. With the rapid development in deep learning, more powerful tools, which are able to learn semantic, high-level, deeper features, are introduced to address the problems existing in traditional architectures. These models behave differently in network architecture, training strategy, and optimization function. In this paper, we provide a review of deep learning-based object detection frameworks. Our review begins with a brief introduction on the history of deep learning and its representative tool, namely, the convolutional neural network. Then, we focus on typical generic object detection architectures along with some modifications and useful tricks to improve detection performance further. As distinct specific detection tasks exhibit different characteristics, we also briefly survey several specific tasks, including salient object detection, face detection, and pedestrian detection. Experimental analyses are also provided to compare various methods and draw some meaningful conclusions. Finally, several promising directions and tasks are provided to serve as guidelines for future work in both object detection and relevant neural network-based learning systems."}59{"work_id": 4388823657, "title": "A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS", "abstract": "YOLO has become a central real-time object detection system for robotics, driverless cars, and video monitoring applications. We present a comprehensive analysis of YOLO\u2019s evolution, examining the innovations and contributions in each iteration from the original YOLO up to YOLOv8, YOLO-NAS, and YOLO with transformers. We start by describing the standard metrics and postprocessing; then, we discuss the major changes in network architecture and training tricks for each model. Finally, we summarize the essential lessons from YOLO\u2019s development and provide a perspective on its future, highlighting potential research directions to enhance real-time object detection systems."}60{"work_id": 4226069476, "title": "Masked-attention Mask Transformer for Universal Image Segmentation", "abstract": "Image segmentation groups pixels with different semantics, e.g., category or instance membership. Each choice of semantics defines a task. While only the semantics of each task differ, current research focuses on designing spe-cialized architectures for each task. We present Masked- attention Mask Transformer (Mask2Former), a new archi-tecture capable of addressing any image segmentation task (panoptic, instance or semantic). Its key components in-clude masked attention, which extracts localized features by constraining cross-attention within predicted mask regions. In addition to reducing the research effort by at least three times, it outperforms the best specialized architectures by a significant margin on four popular datasets. Most no-tably, Mask2Former sets a new state-of-the-art for panoptic segmentation (57.8 PQ on COCO), instance segmentation (50.1 AP on COCO) and semantic segmentation (57.7 mIoU onADE20K)."}61{"work_id": 4224035735, "title": "Hierarchical Text-Conditional Image Generation with CLIP Latents", "abstract": "Contrastive models like CLIP have been shown to learn robust representations of images that capture both semantics and style. To leverage these representations for image generation, we propose a two-stage model: a prior that generates a CLIP image embedding given a text caption, and a decoder that generates an image conditioned on the image embedding. We show that explicitly generating image representations improves image diversity with minimal loss in photorealism and caption similarity. Our decoders conditioned on image representations can also produce variations of an image that preserve both its semantics and style, while varying the non-essential details absent from the image representation. Moreover, the joint embedding space of CLIP enables language-guided image manipulations in a zero-shot fashion. We use diffusion models for the decoder and experiment with both autoregressive and diffusion models for the prior, finding that the latter are computationally more efficient and produce higher-quality samples."}62{"work_id": 4402660132, "title": "Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis", "abstract": "Novel view synthesis of dynamic scenes has been an intriguing yet challenging problem. Despite recent advancements, simultaneously achieving high-resolution photorealistic results, real-time rendering, and compact storage remains a formidable task. To address these challenges, we propose Spacetime Gaussian Feature Splatting as a novel dynamic scene representation, composed of three pivotal components. First, we formulate expressive Spacetime Gaussians by enhancing 3D Gaussians with temporal opacity and parametric motion/rotation. This enables Space-time Gaussians to capture static, dynamic, as well as transient content within a scene. Second, we introduce splatted feature rendering, which replaces spherical harmonics with neural features. These features facilitate the modeling of view- and time-dependent appearance while maintaining small size. Third, we leverage the guidance of training error and coarse depth to sample new Gaussians in areas that are challenging to converge with existing pipelines. Experiments on several established real-world datasets demonstrate that our method achieves state-of-the-art rendering quality and speed, while retaining compact storage. At 8K resolution, our lite-version model can render at 60 FPS on an Nvidia RTX 4090 GPU."}63{"work_id": 2963188159, "title": "PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes", "abstract": "Estimating the 6D pose of known objects is important for robots to interact with the real world.The problem is challenging due to the variety of objects as well as the complexity of a scene caused by clutter and occlusions between objects.In this work, we introduce PoseCNN, a new Convolutional Neural Network for 6D object pose estimation.PoseCNN estimates the 3D translation of an object by localizing its center in the image and predicting its distance from the camera.The 3D rotation of the object is estimated by regressing to a quaternion representation.We also introduce a novel loss function that enables PoseCNN to handle symmetric objects.In addition, we contribute a large scale video dataset for 6D object pose estimation named the YCB-Video dataset.Our dataset provides accurate 6D poses of 21 objects from the YCB dataset observed in 92 videos with 133,827 frames.We conduct extensive experiments on our YCB-Video dataset and the OccludedLINEMOD dataset to show that PoseCNN is highly robust to occlusions, can handle symmetric objects, and provide accurate pose estimation using only color images as input.When using depth data to further refine the poses, our approach achieves state-of-the-art results on the challenging OccludedLINEMOD dataset."}64{"work_id": 4385289063, "title": "MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenes", "abstract": "Neural radiance fields enable state-of-the-art photorealistic view synthesis. However, existing radiance field representations are either too compute-intensive for real-time rendering or require too much memory to scale to large scenes. We present a Memory-Efficient Radiance Field (MERF) representation that achieves real-time rendering of large-scale scenes in a browser. MERF reduces the memory consumption of prior sparse volumetric radiance fields using a combination of a sparse feature grid and high-resolution 2D feature planes. To support large-scale unbounded scenes, we introduce a novel contraction function that maps scene coordinates into a bounded volume while still allowing for efficient ray-box intersection. We design a lossless procedure for baking the parameterization used during training into a model that achieves real-time rendering while still preserving the photorealistic view synthesis quality of a volumetric radiance field."}65{"work_id": 3003533476, "title": "Deep Reinforcement Learning for Autonomous Driving: A Survey", "abstract": "With the development of deep representation learning, the domain of reinforcement learning (RL) has become a powerful learning framework now capable of learning complex policies in high dimensional environments. This review summarises deep reinforcement learning (DRL) algorithms and provides a taxonomy of automated driving tasks where (D)RL methods have been employed, while addressing key computational challenges in real world deployment of autonomous driving agents. It also delineates adjacent domains such as behavior cloning, imitation learning, inverse reinforcement learning that are related but are not classical RL algorithms. The role of simulators in training agents, methods to validate, test and robustify existing solutions in RL are discussed."}66{"work_id": 2796347433, "title": "YOLOv3: An Incremental Improvement", "abstract": "We present some updates to YOLO! We made a bunch of little design changes to make it better. We also trained this new network that's pretty swell. It's a little bigger than last time but more accurate. It's still fast though, don't worry. At 320x320 YOLOv3 runs in 22 ms at 28.2 mAP, as accurate as SSD but three times faster. When we look at the old .5 IOU mAP detection metric YOLOv3 is quite good. It achieves 57.9 mAP@50 in 51 ms on a Titan X, compared to 57.5 mAP@50 in 198 ms by RetinaNet, similar performance but 3.8x faster. As always, all the code is online at https://pjreddie.com/yolo/"}67{"work_id": 2775304348, "title": "State-of-the-Art Speech Recognition with Sequence-to-Sequence Models", "abstract": "Attention-based encoder-decoder architectures such as Listen, Attend, and Spell (LAS), subsume the acoustic, pronunciation and language model components of a traditional automatic speech recognition (ASR) system into a single neural network. In previous work, we have shown that such architectures are comparable to state-of-the-art ASR systems on dictation tasks, but it was not clear if such architectures would be practical for more challenging tasks such as voice search. In this work, we explore a variety of structural and optimization improvements to our LAS model which significantly improve performance. On the structural side, we show that word piece models can be used instead of graphemes. We also introduce a multi-head attention architecture, which offers improvements over the commonly-used single-head attention. On the optimization side, we explore synchronous training, scheduled sampling, label smoothing, and minimum word error rate optimization, which are all shown to improve accuracy. We present results with a unidirectional LSTM encoder for streaming recognition. On a 12, 500 hour voice search task, we find that the proposed changes improve the WER from 9.2% to 5.6%, while the best conventional system achieves 6.7%; on a dictation task our model achieves a WER of 4.1% compared to 5% for the conventional system."}68{"work_id": 3097777922, "title": "Conformer: Convolution-augmented Transformer for Speech Recognition", "abstract": "Recently Transformer and Convolution neural network (CNN) based models have shown promising results in Automatic Speech Recognition (ASR), outperforming Recurrent neural networks (RNNs). Transformer models are good at capturing content-based global interactions, while CNNs exploit local features effectively. In this work, we achieve the best of both worlds by studying how to combine convolution neural networks and transformers to model both local and global dependencies of an audio sequence in a parameter-efficient way. To this regard, we propose the convolution-augmented transformer for speech recognition, named Conformer. Conformer significantly outperforms the previous Transformer and CNN based models achieving state-of-the-art accuracies. On the widely used LibriSpeech benchmark, our model achieves WER of 2.1%/4.3% without using a language model and 1.9%/3.9% with an external language model on test/testother. We also observe competitive performance of 2.7%/6.3% with a small model of only 10M parameters."}69{"work_id": 7140452175, "title": "Humanoid Locomotion and Manipulation: Current Progress and Challenges in Control, Planning, and Learning", "abstract": "Humanoid robots hold great potential to perform various human-level skills, involving unified locomotion and manipulation in real-world settings. Driven by advances in machine learning and the strength of existing model-based approaches, these capabilities have progressed rapidly, but often separately. This survey offers a comprehensive overview of the state-of-the-art in humanoid locomotion and manipulation, with a focus on control, planning, and learning methods. We first review the model-based methods that have been the backbone of humanoid robotics for the past three decades. We discuss contact planning, motion planning, and whole-body control, highlighting the tradeoffs between model fidelity and computational efficiency. Then, the focus is shifted to examine emerging learning-based methods, with an emphasis on reinforcement and imitation learning that enhance the robustness and versatility of loco-manipulation skills. Furthermore, we assess the potential of integrating foundation models with humanoid embodiments to enable the development of generalist humanoid agents. This survey also highlights the emerging role of tactile sensing, particularly whole-body tactile feedback, as a crucial modality for handling contact-rich interactions. Finally, we compare the strengths and limitations of model-based and learning-based paradigms from multiple perspectives, such as robustness, computational efficiency, versatility, and generalizability, and suggest potential solutions to existing challenges."}70{"work_id": 4405586205, "title": "Generative AI in higher education: A global perspective of institutional adoption policies and guidelines", "abstract": "Integrating generative AI (GAI) into higher education is crucial for preparing a future generation of GAI-literate students. However, a comprehensive understanding of global institutional adoption policies remains absent, with most prior studies focusing on the Global North and lacking a theoretical lens. This study utilizes the Diffusion of Innovations Theory to examine GAI adoption strategies in higher education across 40 universities from six global regions. It explores the characteristics of GAI innovation, including compatibility, trialability, and observability, and analyses the communication channels and roles and responsibilities outlined in university policies and guidelines. The findings reveal that universities are proactively addressing GAI integration by emphasising academic integrity, enhancing teaching and learning practices, and promoting equity. Key policy measures include the development of guidelines for ethical GAI use, the design of authentic assessments to mitigate misuse, and the provision of training programs for faculty and students to foster GAI literacy. Despite these efforts, gaps remain in comprehensive policy frameworks, particularly in addressing data privacy concerns and ensuring equitable access to GAI tools. The study underscores the importance of clear communication channels, stakeholder collaboration, and ongoing evaluation to support effective GAI adoption. These insights provide actionable insights for policymakers to craft inclusive, transparent, and adaptive strategies for integrating GAI into higher education."}71{"work_id": 2964243274, "title": "Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions", "abstract": "This paper describes Tacotron 2, a neural network architecture for speech synthesis directly from text. The system is composed of a recurrent sequence-to-sequence feature prediction network that maps character embeddings to mel-scale spectrograms, followed by a modified WaveNet model acting as a vocoder to synthesize time-domain waveforms from those spectrograms. Our model achieves a mean opinion score (MOS) of 4.53 comparable to a MOS of 4.58 for professionally recorded speech. To validate our design choices, we present ablation studies of key components of our system and evaluate the impact of using mel spectrograms as the conditioning input to WaveNet instead of linguistic, duration, and F0 features. We further show that using this compact acoustic intermediate representation allows for a significant reduction in the size of the WaveNet architecture."}72{"work_id": 3131500599, "title": "Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions", "abstract": "Although convolutional neural networks (CNNs) have achieved great success in computer vision, this work investigates a simpler, convolution-free backbone network use-fid for many dense prediction tasks. Unlike the recently-proposed Vision Transformer (ViT) that was designed for image classification specifically, we introduce the Pyramid Vision Transformer (PVT), which overcomes the difficulties of porting Transformer to various dense prediction tasks. PVT has several merits compared to current state of the arts. (1) Different from ViT that typically yields low-resolution outputs and incurs high computational and memory costs, PVT not only can be trained on dense partitions of an image to achieve high output resolution, which is important for dense prediction, but also uses a progressive shrinking pyramid to reduce the computations of large feature maps. (2) PVT inherits the advantages of both CNN and Transformer, making it a unified backbone for various vision tasks without convolutions, where it can be used as a direct replacement for CNN backbones. (3) We validate PVT through extensive experiments, showing that it boosts the performance of many downstream tasks, including object detection, instance and semantic segmentation. For example, with a comparable number of parameters, PVT+RetinaNet achieves 40.4 AP on the COCO dataset, surpassing ResNet50+RetinNet (36.3 AP) by 4.1 absolute AP (see Figure 2). We hope that PVT could, serre as an alternative and useful backbone for pixel-level predictions and facilitate future research."}73{"work_id": 2335728318, "title": "Reading digits in natural images with unsupervised feature learning", "abstract": "Detecting and reading text from natural images is a hard computer vision task that is central to a variety of emerging applications. Related problems like document character recognition have been widely studied by computer vision and machine learning researchers and are virtually solved for practical applications like reading handwritten digits. Reliably recognizing characters in more complex scenes like photographs, however, is far more difficult: the best existing methods lag well behind human performance on the same tasks. In this paper we attack the problem of recognizing digits in a real application using unsupervised feature learning methods: reading house numbers from street level photos. To this end, we introduce a new benchmark dataset for research use containing over 600,000 labeled digits cropped from Street View images. We then demonstrate the difficulty of recognizing these digits when the problem is approached with hand-designed features. Finally, we employ variants of two recently proposed unsupervised feature learning methods and find that they are convincingly superior on our benchmarks. 1"}74{"work_id": 4388761637, "title": "Towards autonomous multi-modal mobility morphobot (M4) robot: traversability estimation and 3D path planning", "abstract": "This thesis enhances the autonomy of the M4 (Multi-Modal Mobility Morphobot) robot,designed for Mars and rescue missions. The research enables the robot to autonomously select its locomotion mode and path in complex terrains. Focusing on walking and flying modes, a Gazebo simulation and custom perception-navigations pipelines are developed. Leveraging deep learning, the robot determines optimal mode transitions based on a 2.5D map. Additionally, an energyefficient path planner based on 2.5D mapping is implemented and validated in simulations. The contributions demonstrate scalability for future mode integrations. The M4 robot showcases intelligent mode switching, efficient navigation, and reduced energy consumption, bringing us closer to fully autonomous multi-modal robots for exploration and rescue missions. This work paves the way for future advancements in autonomous robotics, with the ultimate vision of deploying the M4 robot for exploration and rescue tasks, making a significant impact in the quest for intelligent and versatile robotic systems.--Author's abstract"}75{"work_id": 1530164934, "title": "A Probabilistic Model of Hierarchical Music Analysis", "abstract": "Schenkerian music theory supposes that Western tonal compositions can be viewed as hierarchies of musical objects. The process of Schenkerian analysis reveals this hierarchy by identifying connections between notes or chords of a composition that illustrate both the small- and large-scale construction of the music. We present a new probabilistic model of this variety of music analysis, details of how the parameters of the model can be learned from a corpus, an algorithm for deriving the most probable analysis for a given piece of music, and both quantitative and human-based evaluations of the algorithm's performance. In addition, we describe the creation of the corpus, the first publicly available data set to contain both musical excerpts and corresponding computer-readable Schenkerian analyses. Combining this corpus with the probabilistic model gives us the first completely data-driven computational approach to hierarchical music analysis."}76{"work_id": 3098044990, "title": "Traduction Non Supervis\u00e9e de Langages de Programmation", "abstract": "A transcompiler, also known as source-to-source translator, is a system that converts source code from a high-level programming language (such as C++ or Python) to another. Transcompilers are primarily used for interoperability, and to port codebases written in an obsolete or deprecated language (e.g. COBOL, Python2) to a modern one. They typically rely on handcrafted rewrite rules, applied to the source code abstract syntax tree. Unfortunately, the resulting translations often lack readability, fail to respect the target language conventions, and require manual modifications in order to work properly. The overall translation process is time-consuming and requires expertise in both the source and target languages, making code-translation projects expensive. Although neural models significantly outperform their rule-based counterparts in the context of natural language translation, their applications to transcompilation have been limited due to the scarcity of parallel data in this domain. In this thesis, we propose methods to train effective and fully unsupervised neural transcompilers.Natural language translators are evaluated with metrics based on token co-occurences between the translation and the reference. We identify that they do not capture the semantics of programming languages. Hence, we build and release a test set composed of 852 parallel functions, along with unit tests to check the semantic correctness of translations. We first leverage objectives designed for natural languages to learn multilingual representations of source code, and train a model to translate, using source code from open source GitHub projects. This model outperforms rule-based methods for translating functions between C++, Java, and Python. Then, we develop an improved pre-training method, which leads the model to learn deeper semantic representations of source code. It results in enhanced performances on several tasks including unsupervised code translation. Finally, we use automated unit tests to automatically create examples of program translations. Training on these examples leads to significant improvements in the performance of our neural transcompilers. Our methods rely exclusively on monolingual source code, require no expertise in the source or target languages, and can easily be generalized to other programming languages."}77{"work_id": 3014641072, "title": "Deep High-Resolution Representation Learning for Visual Recognition", "abstract": "High-resolution representations are essential for position-sensitive vision problems, such as human pose estimation, semantic segmentation, and object detection. Existing state-of-the-art frameworks first encode the input image as a low-resolution representation through a subnetwork that is formed by connecting high-to-low resolution convolutions in series (e.g., ResNet, VGGNet), and then recover the high-resolution representation from the encoded low-resolution representation. Instead, our proposed network, named as High-Resolution Network (HRNet), maintains high-resolution representations through the whole process. There are two key characteristics: (i) Connect the high-to-low resolution convolution streams in parallel and (ii) repeatedly exchange the information across resolutions. The benefit is that the resulting representation is semantically richer and spatially more precise. We show the superiority of the proposed HRNet in a wide range of applications, including human pose estimation, semantic segmentation, and object detection, suggesting that the HRNet is a stronger backbone for computer vision problems. All the codes are available at https://github.com/HRNet."}78{"work_id": 2752782242, "title": "Squeeze-and-Excitation Networks", "abstract": "Convolutional neural networks are built upon the convolution operation, which extracts informative features by fusing spatial and channel-wise information together within local receptive fields. In order to boost the representational power of a network, several recent approaches have shown the benefit of enhancing spatial encoding. In this work, we focus on the channel relationship and propose a novel architectural unit, which we term the \"Squeeze-and-Excitation\" (SE) block, that adaptively recalibrates channel-wise feature responses by explicitly modelling interdependencies between channels. We demonstrate that by stacking these blocks together, we can construct SENet architectures that generalise extremely well across challenging datasets. Crucially, we find that SE blocks produce significant performance improvements for existing state-of-the-art deep architectures at minimal additional computational cost. SENets formed the foundation of our ILSVRC 2017 classification submission which won first place and significantly reduced the top-5 error to 2.251%, achieving a ~25% relative improvement over the winning entry of 2016. Code and models are available at https://github.com/hujie-frank/SENet."}79{"work_id": 4294891500, "title": "A Survey on Metaverse: Fundamentals, Security, and Privacy", "abstract": "Metaverse, as an evolving paradigm of the next-generation Internet, aims to build a fully immersive, hyper spatiotemporal, and self-sustaining virtual shared space for humans to play, work, and socialize. Driven by recent advances in emerging technologies such as extended reality, artificial intelligence, and blockchain, metaverse is stepping from science fiction to an upcoming reality. However, severe privacy invasions and security breaches (inherited from underlying technologies or emerged in the new digital ecology) of metaverse can impede its wide deployment. At the same time, a series of fundamental challenges (e.g., scalability and interoperability) can arise in metaverse security provisioning owing to the intrinsic characteristics of metaverse, such as immersive realism, hyper spatiotemporality, sustainability, and heterogeneity. In this paper, we present a comprehensive survey of the fundamentals, security, and privacy of metaverse. Specifically, we first investigate a novel distributed metaverse architecture and its key characteristics with ternary-world interactions. Then, we discuss the security and privacy threats, present the critical challenges of metaverse systems, and review the state-of-the-art countermeasures. Finally, we draw open research directions for building future metaverse systems."}80{"work_id": 3005680577, "title": "A Simple Framework for Contrastive Learning of Visual Representations", "abstract": "This paper presents SimCLR: a simple framework for contrastive learning of visual representations. We simplify recently proposed contrastive self-supervised learning algorithms without requiring specialized architectures or a memory bank. In order to understand what enables the contrastive prediction tasks to learn useful representations, we systematically study the major components of our framework. We show that (1) composition of data augmentations plays a critical role in defining effective predictive tasks, (2) introducing a learnable nonlinear transformation between the representation and the contrastive loss substantially improves the quality of the learned representations, and (3) contrastive learning benefits from larger batch sizes and more training steps compared to supervised learning. By combining these findings, we are able to considerably outperform previous methods for self-supervised and semi-supervised learning on ImageNet. A linear classifier trained on self-supervised representations learned by SimCLR achieves 76.5% top-1 accuracy, which is a 7% relative improvement over previous state-of-the-art, matching the performance of a supervised ResNet-50. When fine-tuned on only 1% of the labels, we achieve 85.8% top-5 accuracy, outperforming AlexNet with 100X fewer labels."}81{"work_id": 4392772072, "title": "AI in education: A review of personalized learning and educational technology", "abstract": "The integration of Artificial Intelligence (AI) in education has ushered in a transformative era, redefining traditional teaching and learning methods. This review explores the multifaceted role of AI in education, with a particular focus on personalized learning and educational technology. The synergy between AI and education promises to address individualized needs, enhance student engagement, and optimize learning outcomes. Personalized learning, enabled by AI algorithms, tailors educational experiences to the unique needs, preferences, and pace of each student. This approach goes beyond a one-size-fits-all model, fostering a more inclusive and effective learning environment. The review delves into the diverse applications of AI-driven personalized learning, ranging from adaptive content delivery and real-time feedback to intelligent tutoring systems. It analyzes the impact of these technologies on student performance, highlighting the potential to narrow educational gaps and cater to diverse learning styles. Educational technology, powered by AI, extends beyond the classroom, encompassing online platforms, virtual reality, and interactive tools. The review explores the integration of AI in curriculum development, content creation, and assessment methods, offering insights into how these technologies augment the teaching and learning experience. Furthermore, the review examines the role of AI in automating administrative tasks, allowing educators to redirect their focus towards personalized instruction. Challenges and ethical considerations associated with the adoption of AI in education are also scrutinized. Privacy concerns, algorithmic biases, and the digital divide are discussed, emphasizing the importance of responsible AI implementation. The review underscores the need for collaborative efforts among educators, policymakers, and technologists to establish ethical guidelines and ensure the equitable distribution of AI-enhanced educational resources. This review provides a comprehensive examination of the evolving landscape of AI in education, with a spotlight on personalized learning and educational technology. As the symbiosis between AI and education continues to evolve, this synthesis of current research and trends aims to guide future developments, fostering an informed and progressive approach to the integration of AI in the educational sphere."}82{"work_id": 4398756379, "title": "YOLOv8: A Novel Object Detection Algorithm with Enhanced Performance and Robustness", "abstract": "In recent years, the You Only Look Once (YOLO) series of object detection algorithms have garnered significant attention for their speed and accuracy in real-time applications. This paper presents YOLOv8, a novel object detection algorithm that builds upon the advancements of previous iterations, aiming to further enhance performance and robustness. Inspired by the evolution of YOLO architectures from YOLOv1 to YOLOv7, as well as insights from comparative analyses of models like YOLOv5 and YOLOv6, YOLOv8 incorporates key innovations to achieve optimal speed and accuracy. Leveraging attention mechanisms and dynamic convolution, YOLOv8 introduces improvements specifically tailored for small object detection, addressing challenges highlighted in YOLOv7. Additionally, the integration of voice recognition techniques enhances the algorithm's capabilities for video-based object detection, as demonstrated in YOLOv7. The proposed algorithm undergoes rigorous evaluation against state-of-the-art benchmarks, showcasing superior performance in terms of both detection accuracy and computational efficiency. Experimental results on various datasets confirm the effectiveness of YOLOv8 across diverse scenarios, further validating its suitability for real-world applications. This paper contributes to the ongoing advancements in object detection research by presenting YOLOv8 as a versatile and high-performing algorithm, poised to address the evolving needs of computer vision systems."}83{"work_id": 4229005866, "title": "Analysing Off-The-Shelf Options for Question Answering with Portuguese FAQs", "abstract": "Following the current interest in developing automatic question answering systems, we analyse alternative approaches for finding suitable answers from a list of Frequently Asked Questions (FAQs), in Portuguese. These rely on different technologies, some more established and others more recent, and are all easily adaptable to new lists of FAQs, on new domains. We analyse the effort required for their configuration, the accuracy of their answers, and the time they take to get such answers. We conclude that traditional Information Retrieval (IR) can be a solution for smaller lists of FAQs, but approaches based on deep neural networks for sentence encoding are at least as reliable and less dependent on the number and complexity of the FAQs. We also contribute with a small dataset of Portuguese FAQs on the domain of telecommunications, which was used in our experiments."}84{"work_id": 3167533889, "title": "SpeechBrain: A General-Purpose Speech Toolkit", "abstract": "SpeechBrain is an open-source and all-in-one speech toolkit. It is designed to facilitate the research and development of neural speech processing technologies by being simple, flexible, user-friendly, and well-documented. This paper describes the core architecture designed to support several tasks of common interest, allowing users to naturally conceive, compare and share novel speech processing pipelines. SpeechBrain achieves competitive or state-of-the-art performance in a wide range of speech benchmarks. It also provides training recipes, pretrained models, and inference scripts for popular speech datasets, as well as tutorials which allow anyone with basic Python proficiency to familiarize themselves with speech technologies."}85{"work_id": 4404112583, "title": "All One Needs to Know about Metaverse: A Complete Survey on Technological Singularity, Virtual Ecosystem, and Research Agenda", "abstract": "Since the popularisation of the Internet in the 1990s, the cyberspace has kept evolving. We have created various computer-mediated virtual environments, including social networks, video conferencing, virtual 3D worlds (e.g., VR Chat), augmented reality applications (e.g., Pok\u00e9mon Go), and Non-Fungible Token Games (e.g., Upland). Such virtual environments, albeit non-perpetual and unconnected, have brought us various degrees of digital transformation. The term \u201cmetaverse\u201d has been coined to facilitate further digital transformation in every aspect of our physical lives. At the core of the metaverse stands the vision of an immersive Internet as a gigantic, unified, persistent, and shared realm. While the metaverse may seem futuristic, catalyzed by emerging technologies such as Extended Reality, 5G, and Artificial Intelligence, the digital \u201cbig bang\u201d of our cyberspace is not far away. This survey presents the first effort to offer a comprehensive framework that examines the latest metaverse development under the dimensions of state-of-the-art technologies and metaverse ecosystems and illustrates the possibility of the digital \u201cbig bang\u201d. It is essential to highlight that the metaverse encompasses diverse technologies and ecosystems, calling it an interdisciplinary and emerging field. Its primary objective is to provide users with satisfactory and interactive experiences. First, technologies are the enablers that drive the transition from the current Internet to the metaverse. We thus examine eight enabling technologies rigorously \u2013 Extended Reality, User Interactivity (Human-Computer Interaction), Artificial Intelligence, Blockchain, Computer Vision, IoT and Robotics, Edge and Cloud computing, and Future Mobile Networks. In terms of applications, the metaverse ecosystem allows human users to live and play within a self-sustaining, persistent, and shared realm. Therefore, we discuss six user-centric factors \u2013 Avatar, Content Creation, Virtual Economy, Social Acceptability, Security and Privacy, and Trust and Accountability. Finally, we propose a concrete research agenda for developing the metaverse."}86{"work_id": 4409885395, "title": "The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers", "abstract": "The rise of Generative AI (GenAI) in knowledge workflows raises questions about its impact on critical thinking skills and practices.We survey 319 knowledge workers to investigate 1) when and how they perceive the enaction of critical thinking when using GenAI, and 2) when and why GenAI affects their effort to do so.Participants shared 936 first-hand examples of using GenAI in work tasks.Quantitatively, when considering both task-and user-specific factors, a user's task-specific self-confidence and confidence in GenAI are predictive of whether critical thinking is enacted and the effort of doing so in GenAI-assisted tasks.Specifically, higher confidence in GenAI is associated with less critical thinking, while higher self-confidence is associated with more critical thinking.Qualitatively, GenAI shifts the nature of critical thinking toward information verification, response integration, and task stewardship.Our insights reveal new design challenges and opportunities for developing GenAI tools for knowledge work."}87{"work_id": 3135367836, "title": "Learning Transferable Visual Models From Natural Language Supervision", "abstract": "State-of-the-art computer vision systems are trained to predict a fixed set of predetermined object categories. This restricted form of supervision limits their generality and usability since additional labeled data is needed to specify any other visual concept. Learning directly from raw text about images is a promising alternative which leverages a much broader source of supervision. We demonstrate that the simple pre-training task of predicting which caption goes with which image is an efficient and scalable way to learn SOTA image representations from scratch on a dataset of 400 million (image, text) pairs collected from the internet. After pre-training, natural language is used to reference learned visual concepts (or describe new ones) enabling zero-shot transfer of the model to downstream tasks. We study the performance of this approach by benchmarking on over 30 different existing computer vision datasets, spanning tasks such as OCR, action recognition in videos, geo-localization, and many types of fine-grained object classification. The model transfers non-trivially to most tasks and is often competitive with a fully supervised baseline without the need for any dataset specific training. For instance, we match the accuracy of the original ResNet-50 on ImageNet zero-shot without needing to use any of the 1.28 million training examples it was trained on. We release our code and pre-trained model weights at https://github.com/OpenAI/CLIP."}88{"work_id": 2626778328, "title": "Attention Is All You Need", "abstract": "The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data."}89{"work_id": 3015199127, "title": "Dual-Path RNN: Efficient Long Sequence Modeling for Time-Domain Single-Channel Speech Separation", "abstract": "Recent studies in deep learning-based speech separation have proven the superiority of time-domain approaches to conventional time-frequency-based methods. Unlike the time-frequency domain approaches, the time-domain separation systems often receive input sequences consisting of a huge number of time steps, which introduces challenges for modeling extremely long sequences. Conventional recurrent neural networks (RNNs) are not effective for modeling such long sequences due to optimization difficulties, while one-dimensional convolutional neural networks (1-D CNNs) cannot perform utterance-level sequence modeling when its receptive field is smaller than the sequence length. In this paper, we propose dual-path recurrent neural network (DPRNN), a simple yet effective method for organizing RNN layers in a deep structure to model extremely long sequences. DPRNN splits the long sequential input into smaller chunks and applies intra- and inter-chunk operations iteratively, where the input length can be made proportional to the square root of the original sequence length in each operation. Experiments show that by replacing 1-D CNN with DPRNN and apply sample-level modeling in the time-domain audio separation network (TasNet), a new state-of-the-art performance on WSJ0-2mix is achieved with a 20 times smaller model than the previous best system."}90{"work_id": 4415795831, "title": "YOLOv10: Real-Time End-to-End Object Detection", "abstract": "Over the past years, YOLOs have emerged as the predominant paradigm in the field of real-time object detection owing to their effective balance between computational cost and detection performance. Researchers have explored the architectural designs, optimization objectives, data augmentation strategies, and others for YOLOs, achieving notable progress. However, the reliance on the non-maximum suppression (NMS) for post-processing hampers the end-to-end deployment of YOLOs and adversely impacts the inference latency. Besides, the design of various components in YOLOs lacks the comprehensive and thorough inspection, resulting in noticeable computational redundancy and limiting the model's capability. It renders the suboptimal efficiency, along with considerable potential for performance improvements. In this work, we aim to further advance the performance-efficiency boundary of YOLOs from both the post-processing and model architecture. To this end, we first present the consistent dual assignments for NMS-free training of YOLOs, which brings competitive performance and low inference latency simultaneously. Moreover, we introduce the holistic efficiency-accuracy driven model design strategy for YOLOs. We comprehensively optimize various components of YOLOs from both efficiency and accuracy perspectives, which greatly reduces the computational overhead and enhances the capability. The outcome of our effort is a new generation of YOLO series for real-time end-to-end object detection, dubbed YOLOv10. Extensive experiments show that YOLOv10 achieves state-of-the-art performance and efficiency across various model scales. For example, our YOLOv10-S is 1.8$\\times$ faster than RT-DETR-R18 under the similar AP on COCO, meanwhile enjoying 2.8$\\times$ smaller number of parameters and FLOPs. Compared with YOLOv9-C, YOLOv10-B has 46\\% less latency and 25\\% fewer parameters for the same performance."}91{"work_id": 4414079054, "title": "\u03c0\u2080: A Vision-Language-Action Flow Model for General Robot Control", "abstract": "&&0'% ) 8::!7 2/ .()/) 'GG) Z)0./GSO/,SGL ./O/), G m l hg/%('(G0 /)S/,G% y&/%,)p.n 2.G .G,)|{z 0 /) G2G.G.G 0%G) 2.G .(\u00b02G)\u00a9\u00afSO/,SG )\u00a1 G ,)S\u00b3)S/.G\u00b1 O') O%G 20 )O/ %G O()(./0G.G /2G)2/20/.%/,),pG /. )%G(/ S G)0/&&GG \u00c0GG2) O%G GS2 \u00b3),.\u00b3G.G ) O%G 20 )pG%& &% G)O/ \u00d5(p\u00d3'% \u00b3)2/ .(),p(p),G G. \u00b3"}92{"work_id": 4311000453, "title": "Robust Speech Recognition via Large-Scale Weak Supervision", "abstract": "We study the capabilities of speech processing systems trained simply to predict large amounts of transcripts of audio on the internet. When scaled to 680,000 hours of multilingual and multitask supervision, the resulting models generalize well to standard benchmarks and are often competitive with prior fully supervised results but in a zero-shot transfer setting without the need for any fine-tuning. When compared to humans, the models approach their accuracy and robustness. We are releasing models and inference code to serve as a foundation for further work on robust speech processing."}93{"work_id": 2963125010, "title": "ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices", "abstract": "We introduce an extremely computation-efficient CNN architecture named ShuffleNet, which is designed specially for mobile devices with very limited computing power (e.g., 10-150 MFLOPs). The new architecture utilizes two new operations, pointwise group convolution and channel shuffle, to greatly reduce computation cost while maintaining accuracy. Experiments on ImageNet classification and MS COCO object detection demonstrate the superior performance of ShuffleNet over other structures, e.g. lower top-1 error (absolute 7.8%) than recent MobileNet [12] on ImageNet classification task, under the computation budget of 40 MFLOPs. On an ARM-based mobile device, ShuffleNet achieves ~13\u00d7 actual speedup over AlexNet while maintaining comparable accuracy."}94{"work_id": 2885324189, "title": "Neural Importance Sampling", "abstract": "We propose to use deep neural networks for generating samples in Monte Carlo integration. Our work is based on non-linear independent components estimation (NICE), which we extend in numerous ways to improve performance and enable its application to integration problems. First, we introduce piecewise-polynomial coupling transforms that greatly increase the modeling power of individual coupling layers. Second, we propose to preprocess the inputs of neural networks using one-blob encoding, which stimulates localization of computation and improves inference. Third, we derive a gradient-descent-based optimization for the Kullback-Leibler and the \u03c7 2 divergence for the specific application of Monte Carlo integration with unnormalized stochastic estimates of the target distribution. Our approach enables fast and accurate inference and efficient sample generation independently of the dimensionality of the integration domain. We show its benefits on generating natural images and in two applications to light-transport simulation: first, we demonstrate learning of joint path-sampling densities in the primary sample space and importance sampling of multi-dimensional path prefixes thereof. Second, we use our technique to extract conditional directional densities driven by the product of incident illumination and the BSDF in the rendering equation, and we leverage the densities for path guiding. In all applications, our approach yields on-par or higher performance than competing techniques at equal sample count."}95{"work_id": 4383108457, "title": "ProgPrompt: Generating Situated Robot Task Plans using Large Language Models", "abstract": "Task planning can require defining myriad domain knowledge about the world in which a robot needs to act. To ameliorate that effort, large language models (LLMs) can be used to score potential next actions during task planning, and even generate action sequences directly, given an instruction in natural language with no additional domain information. However, such methods either require enumerating all possible next steps for scoring, or generate free-form text that may contain actions not possible on a given robot in its current context. We present a programmatic LLM prompt structure that enables plan generation functional across situated environments, robot capabilities, and tasks. Our key insight is to prompt the LLM with program-like specifications of the available actions and objects in an environment, as well as with example programs that can be executed. We make concrete recommendations about prompt structure and generation constraints through ablation experiments, demonstrate state of the art success rates in VirtualHome household tasks, and deploy our method on a physical robot arm for tabletop tasks. Website at progprompt.github.io"}96{"work_id": 7125799258, "title": "Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment", "abstract": "With the continuous growth in the number of parameters of the Transformer-based pretrained language models (PLMs), particularly the emergence of large language models (LLMs) with billions of parameters, many natural language processing (NLP) tasks have demonstrated remarkable success. However, the enormous size and computational demands of these models pose significant challenges for adapting them to specific downstream tasks, especially in environments with limited computational resources. Parameter-Efficient Fine-Tuning (PEFT) offers an effective solution by reducing the number of fine-tuning parameters and memory usage while achieving comparable performance to full fine-tuning. The demands for fine-tuning PLMs, especially LLMs, have led to a surge in the development of PEFT methods, as depicted in Fig. 1. In this paper, we present a comprehensive and systematic review of PEFT methods for PLMs. We summarize these PEFT methods, discuss their applications, and outline future directions. Furthermore, extensive experiments are conducted using several representative PEFT methods to better understand their effectiveness in parameter efficiency and memory efficiency. By offering insights into the latest advancements and practical applications, this survey serves as an invaluable resource for researchers and practitioners seeking to navigate the challenges and opportunities presented by PEFT in the context of PLMs."}97{"work_id": 3115857147, "title": "Illumination Space: A Feature Space for Radiance Maps", "abstract": "Radiance maps (RM) are used for capturing the lighting properties of real-world environments. Databases of RMs are useful for various rendering applications such as Look Development, live action composition, mixed reality, and machine learning. Such databases are not useful if they cannot be organized in a meaningful way. To address this, we introduce the illumination space, a feature space that arranges RM databases based on illumination properties. We avoid manual labeling by automatically extracting features from an RM that provides a concise and semantically meaningful representation of its typical lighting effects. This is made possible with the following contributions: a method to automatically extract a small set of dominant and ambient lighting properties from RMs, and a low-dimensional (5D) light feature vector summarizing these properties to form the illumination space. Our method is motivated by how the RM illuminates the scene as opposed to describing the textural content of the RM."}98{"work_id": 2978950353, "title": "PERANCANGAN POP UP BOOK AND SOUND TEMBANG DOLANAN SEBAGAI MEDIA PENGENALAN ALAT MUSIK GAMELAN", "abstract": "Perancangan Pop Up Book and Sound Tembang Dolanan Sebagai Pengenalan Alat Musik Gamelan. NAMA: Ilham Musyafa Asya Ari NIM. 12151125. Program Studi S-1 Desain Komunikasii Visual. Kesenian tradisional Jawa banyak macamnya, salah satunya adalah seni tembang dolanan yang diiringi dengan gamelan jawa. Saat ini dalam mata pelajaran sekolah siswa SD (sekolah Dasar) di Surakarta sangat jarang diperkenalkan, karena pada kenyataannya instrument gamelan tidak dimiliki atau jarang ada sebagai laboratorim pembelajaran. Pembelajaran tembang dolanan diiringi instrumen gamelan dianggap kurang menarik dan ketinggalan jaman, meskipun ada instrument campursari atau organ tunggal. Perancangan buku pop up book tembang dolanan ini diciptakan, sebagai media ajar yang baru bertujuan agar lebih menarik generasi masa kini terutama anak-anak sekolah dasar untuk mengenal kembali kebudayaan musik tradisi yang sarat akan makna yang keberadaannya sudah mulai dilupakan. Metode penciptaan ini dimulai dengan riset dan wawancara ke sekolah-sekolah dasar di Surakarta, melakukan brainstorming, eksplorasi ide, penciptaan naskah, sketsa, layout, dummy, dan artwork. Media promosi untuk memperkenalkan Pop up book ini melalui media lini atas yaitu website atau media sosial dan media pendukung seperti pensil warna, totegbag, pin dan sticker, sebagai bentuk untuk menyadarkan arti pentingnya mencintai budaya lokal pada tembang dolanan."}99{"work_id": 4212875960, "title": "UNETR: Transformers for 3D Medical Image Segmentation", "abstract": "Fully Convolutional Neural Networks (FCNNs) with contracting and expanding paths have shown prominence for the majority of medical image segmentation applications since the past decade. In FCNNs, the encoder plays an integral role by learning both global and local features and contextual representations which can be utilized for semantic output prediction by the decoder. Despite their success, the locality of convolutional layers in FCNNs, limits the capability of learning long-range spatial dependencies. Inspired by the recent success of transformers for Natural Language Processing (NLP) in long-range sequence learning, we reformulate the task of volumetric (3D) medical image segmentation as a sequence-to-sequence prediction problem. We introduce a novel architecture, dubbed as UNEt TRansformers (UNETR), that utilizes a transformer as the encoder to learn sequence representations of the input volume and effectively capture the global multi-scale information, while also following the successful \"U-shaped\" network design for the encoder and decoder. The transformer encoder is directly connected to a decoder via skip connections at different resolutions to compute the final semantic segmentation output. We have validated the performance of our method on the Multi Atlas Labeling Beyond The Cranial Vault (BTCV) dataset for multi-organ segmentation and the Medical Segmentation Decathlon (MSD) dataset for brain tumor and spleen segmentation tasks. Our benchmarks demonstrate new state-of-the-art performance on the BTCV leaderboard."}100{"work_id": 4400881081, "title": "TransUNet: Rethinking the U-Net architecture design for medical image segmentation through the lens of transformers", "abstract": "Medical image segmentation is crucial for healthcare, yet convolution-based methods like U-Net face limitations in modeling long-range dependencies. To address this, Transformers designed for sequence-to-sequence predictions have been integrated into medical image segmentation. However, a comprehensive understanding of Transformers' self-attention in U-Net components is lacking. TransUNet, first introduced in 2021, is widely recognized as one of the first models to integrate Transformer into medical image analysis. In this study, we present the versatile framework of TransUNet that encapsulates Transformers' self-attention into two key modules: (1) a Transformer encoder tokenizing image patches from a convolution neural network (CNN) feature map, facilitating global context extraction, and (2) a Transformer decoder refining candidate regions through cross-attention between proposals and U-Net features. These modules can be flexibly inserted into the U-Net backbone, resulting in three configurations: Encoder-only, Decoder-only, and Encoder+Decoder. TransUNet provides a library encompassing both 2D and 3D implementations, enabling users to easily tailor the chosen architecture. Our findings highlight the encoder's efficacy in modeling interactions among multiple abdominal organs and the decoder's strength in handling small targets like tumors. It excels in diverse medical applications, such as multi-organ segmentation, pancreatic tumor segmentation, and hepatic vessel segmentation. Notably, our TransUNet achieves a significant average Dice improvement of 1.06% and 4.30% for multi-organ segmentation and pancreatic tumor segmentation, respectively, when compared to the highly competitive nn-UNet, and surpasses the top-1 solution in the BrasTS2021 challenge. 2D/3D Code and models are available at https://github.com/Beckschen/TransUNet and https://github.com/Beckschen/TransUNet-3D, respectively."}101{"work_id": 7135186946, "title": "Scalable Fluid Antenna Systems: A New Paradigm for Array Signal Processing", "abstract": "Most existing antenna array-based source localization methods rely on fixed-position arrays (FPAs) and strict assumptions about source field conditions (near-field or far-field), which limits their effectiveness in complex, dynamic real-world scenarios where high-precision localization is required. In contrast, this paper introduces a novel scalable fluid antenna system (SFAS) that can dynamically adjust its aperture configuration to optimize performance for different localization tasks. Within this framework, we develop a two-stage source localization strategy based on the exact spatial geometry (ESG) model: the first stage uses a compact aperture configuration for initial direction-of-arrival (DOA) estimation, while the second stage employs an expanded aperture for enhanced DOA and range estimation. The proposed approach eliminates the traditional need for signal separation or isolation to classify source types and enables a single SFAS array to achieve high localization accuracy without field-specific assumptions, model simplifications, or approximations, representing a new paradigm in array-based source localization. Extensive simulations demonstrate the superiority of the proposed method in terms of localization accuracy, computational efficiency, and robustness to different source types."}102{"work_id": 2963163009, "title": "MobileNetV2: Inverted Residuals and Linear Bottlenecks", "abstract": "In this paper we describe a new mobile architecture, MobileNetV2, that improves the state of the art performance of mobile models on multiple tasks and benchmarks as well as across a spectrum of different model sizes. We also describe efficient ways of applying these mobile models to object detection in a novel framework we call SSDLite. Additionally, we demonstrate how to build mobile semantic segmentation models through a reduced form of DeepLabv3 which we call Mobile DeepLabv3. is based on an inverted residual structure where the shortcut connections are between the thin bottleneck layers. The intermediate expansion layer uses lightweight depthwise convolutions to filter features as a source of non-linearity. Additionally, we find that it is important to remove non-linearities in the narrow layers in order to maintain representational power. We demonstrate that this improves performance and provide an intuition that led to this design. Finally, our approach allows decoupling of the input/output domains from the expressiveness of the transformation, which provides a convenient framework for further analysis. We measure our performance on ImageNet [1] classification, COCO object detection [2], VOC image segmentation [3]. We evaluate the trade-offs between accuracy, and number of operations measured by multiply-adds (MAdd), as well as actual latency, and the number of parameters."}103{"work_id": 2999607073, "title": "Image Segmentation Using Deep Learning: A Survey", "abstract": "Image segmentation is a key task in computer vision and image processing with important applications such as scene understanding, medical image analysis, robotic perception, video surveillance, augmented reality, and image compression, among others, and numerous segmentation algorithms are found in the literature. Against this backdrop, the broad success of deep learning (DL) has prompted the development of new image segmentation approaches leveraging DL models. We provide a comprehensive review of this recent literature, covering the spectrum of pioneering efforts in semantic and instance segmentation, including convolutional pixel-labeling networks, encoder-decoder architectures, multiscale and pyramid-based approaches, recurrent networks, visual attention models, and generative models in adversarial settings. We investigate the relationships, strengths, and challenges of these DL-based segmentation models, examine the widely used datasets, compare performances, and discuss promising research directions."}104{"work_id": 4406918931, "title": "UAV swarms: research, challenges, and future directions", "abstract": "Abstract Unmanned Aerial Vehicle (UAV) swarms represent a transformative advancement in aerial robotics, leveraging collaborative autonomy to enhance operational capabilities. This paper provides a comprehensive exploration of UAV swarm infrastructure, recent research advancements, and diverse applications. Key areas such as coordinated path planning, task assignment, formation control, and security considerations are examined, highlighting how Artificial Intelligence (AI) and Machine Learning (ML) are integrated to improve decision-making and adaptability. Applications span civilian sectors, including entertainment, infrastructure inspection, and delivery services, as well as military applications in surveillance, combat support, and logistics. The paper addresses technical challenges, regulatory constraints, and ethical considerations, while outlining future directions focused on scalability, robustness, and societal integration. This review consolidates the evolving landscape of UAV swarms, identifying critical challenges and guiding future research endeavors."}105{"work_id": 2925329277, "title": "Control Barrier Functions: Theory and Applications", "abstract": "This paper provides an introduction and overview of recent work on control barrier functions and their use to verify and enforce safety properties in the context of (optimization based) safety-critical controllers. We survey the main technical results and discuss applications to several domains including robotic systems."}106{"work_id": 2978017171, "title": "DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter", "abstract": "As Transfer Learning from large-scale pre-trained models becomes more prevalent in Natural Language Processing (NLP), operating these large models in on-the-edge and/or under constrained computational training or inference budgets remains challenging. In this work, we propose a method to pre-train a smaller general-purpose language representation model, called DistilBERT, which can then be fine-tuned with good performances on a wide range of tasks like its larger counterparts. While most prior work investigated the use of distillation for building task-specific models, we leverage knowledge distillation during the pre-training phase and show that it is possible to reduce the size of a BERT model by 40%, while retaining 97% of its language understanding capabilities and being 60% faster. To leverage the inductive biases learned by larger models during pre-training, we introduce a triple loss combining language modeling, distillation and cosine-distance losses. Our smaller, faster and lighter model is cheaper to pre-train and we demonstrate its capabilities for on-device computations in a proof-of-concept experiment and a comparative on-device study."}107{"work_id": 3156636935, "title": "SimCSE: Simple Contrastive Learning of Sentence Embeddings", "abstract": "This paper presents SimCSE, a simple contrastive learning framework that greatly advances the state-of-the-art sentence embeddings.We first describe an unsupervised approach, which takes an input sentence and predicts itself in a contrastive objective, with only standard dropout used as noise.This simple method works surprisingly well, performing on par with previous supervised counterparts.We find that dropout acts as minimal data augmentation and removing it leads to a representation collapse.Then, we propose a supervised approach, which incorporates annotated pairs from natural language inference datasets into our contrastive learning framework, by using \"entailment\" pairs as positives and \"contradiction\" pairs as hard negatives.We evaluate SimCSE on standard semantic textual similarity (STS) tasks, and our unsupervised and supervised models using BERT base achieve an average of 76.3% and 81.6% Spearman's correlation respectively, a 4.2% and 2.2% improvement compared to previous best results.We also show-both theoretically and empirically-that contrastive learning objective regularizes pre-trained embeddings' anisotropic space to be more uniform, and it better aligns positive pairs when supervised signals are available.1"}108{"work_id": 4391109864, "title": "Segment anything in medical images", "abstract": "Medical image segmentation is a critical component in clinical practice, facilitating accurate diagnosis, treatment planning, and disease monitoring. However, existing methods, often tailored to specific modalities or disease types, lack generalizability across the diverse spectrum of medical image segmentation tasks. Here we present MedSAM, a foundation model designed for bridging this gap by enabling universal medical image segmentation. The model is developed on a large-scale medical image dataset with 1,570,263 image-mask pairs, covering 10 imaging modalities and over 30 cancer types. We conduct a comprehensive evaluation on 86 internal validation tasks and 60 external validation tasks, demonstrating better accuracy and robustness than modality-wise specialist models. By delivering accurate and efficient segmentation across a wide spectrum of tasks, MedSAM holds significant potential to expedite the evolution of diagnostic tools and the personalization of treatment plans."}109{"work_id": 2962866211, "title": "Supervised Speech Separation Based on Deep Learning: An Overview", "abstract": "Speech separation is the task of separating target speech from background interference. Traditionally, speech separation is studied as a signal processing problem. A more recent approach formulates speech separation as a supervised learning problem, where the discriminative patterns of speech, speakers, and background noise are learned from training data. Over the past decade, many supervised separation algorithms have been put forward. In particular, the recent introduction of deep learning to supervised speech separation has dramatically accelerated progress and boosted separation performance. This paper provides a comprehensive overview of the research on deep learning based supervised speech separation in the last several years. We first introduce the background of speech separation and the formulation of supervised separation. Then, we discuss three main components of supervised separation: learning machines, training targets, and acoustic features. Much of the overview is on separation algorithms where we review monaural methods, including speech enhancement (speech-nonspeech separation), speaker separation (multitalker separation), and speech dereverberation, as well as multimicrophone techniques. The important issue of generalization, unique to supervised learning, is discussed. This overview provides a historical perspective on how advances are made. In addition, we discuss a number of conceptual issues, including what constitutes the target source."}110{"work_id": 4221151978, "title": "Instant neural graphics primitives with a multiresolution hash encoding", "abstract": "Neural graphics primitives, parameterized by fully connected neural networks, can be costly to train and evaluate. We reduce this cost with a versatile new input encoding that permits the use of a smaller network without sacrificing quality, thus significantly reducing the number of floating point and memory access operations: a small neural network is augmented by a multiresolution hash table of trainable feature vectors whose values are optimized through stochastic gradient descent. The multiresolution structure allows the network to disambiguate hash collisions, making for a simple architecture that is trivial to parallelize on modern GPUs. We leverage this parallelism by implementing the whole system using fully-fused CUDA kernels with a focus on minimizing wasted bandwidth and compute operations. We achieve a combined speedup of several orders of magnitude, enabling training of high-quality neural graphics primitives in a matter of seconds, and rendering in tens of milliseconds at a resolution of 1920\u00d71080."}111{"work_id": 2904367110, "title": "A Style-Based Generator Architecture for Generative Adversarial Networks", "abstract": "We propose an alternative generator architecture for generative adversarial networks, borrowing from style transfer literature. The new architecture leads to an automatically learned, unsupervised separation of high-level attributes (e.g., pose and identity when trained on human faces) and stochastic variation in the generated images (e.g., freckles, hair), and it enables intuitive, scale-specific control of the synthesis. The new generator improves the state-of-the-art in terms of traditional distribution quality metrics, leads to demonstrably better interpolation properties, and also better disentangles the latent factors of variation. To quantify interpolation quality and disentanglement, we propose two new, automated methods that are applicable to any generator architecture. Finally, we introduce a new, highly varied and high-quality dataset of human faces."}112{"work_id": 4390723081, "title": "A Survey on 3D Gaussian Splatting", "abstract": "3D Gaussian splatting (GS) has emerged as a transformative technique in radiance fields. Unlike mainstream implicit neural models, 3D GS uses millions of learnable 3D Gaussians for an explicit scene representation. Paired with a differentiable rendering algorithm, this approach achieves real-time rendering and unprecedented editability, making it a potential game-changer for 3D reconstruction and representation. In the present article, we provide the first systematic overview of the recent developments and critical contributions in 3D GS. We begin with a detailed exploration of the underlying principles and the driving forces behind the emergence of 3D GS, laying the groundwork for understanding its significance. A focal point of our discussion is the practical applicability of 3D GS. By enabling unprecedented rendering speed, 3D GS opens up a plethora of applications, ranging from virtual reality to interactive media and beyond. This is complemented by a comparative analysis of leading 3D GS models, evaluated across various benchmark tasks to highlight their performance and practical utility. The survey concludes by identifying current challenges and suggesting potential avenues for future research. Through this survey, we aim to provide a valuable resource for both newcomers and seasoned researchers, fostering further exploration and advancement in explicit radiance field."}113{"work_id": 4323572061, "title": "PaLM-E: An Embodied Multimodal Language Model", "abstract": "Large language models excel at a wide range of complex tasks. However, enabling general inference in the real world, e.g., for robotics problems, raises the challenge of grounding. We propose embodied language models to directly incorporate real-world continuous sensor modalities into language models and thereby establish the link between words and percepts. Input to our embodied language model are multi-modal sentences that interleave visual, continuous state estimation, and textual input encodings. We train these encodings end-to-end, in conjunction with a pre-trained large language model, for multiple embodied tasks including sequential robotic manipulation planning, visual question answering, and captioning. Our evaluations show that PaLM-E, a single large embodied multimodal model, can address a variety of embodied reasoning tasks, from a variety of observation modalities, on multiple embodiments, and further, exhibits positive transfer: the model benefits from diverse joint training across internet-scale language, vision, and visual-language domains. Our largest model, PaLM-E-562B with 562B parameters, in addition to being trained on robotics tasks, is a visual-language generalist with state-of-the-art performance on OK-VQA, and retains generalist language capabilities with increasing scale."}114{"work_id": 3196350727, "title": "HISTORIAE, History of Socio-Cultural Transformation as Linguistic Data Science. A Humanities Use Case", "abstract": "The paper proposes an interdisciplinary approach including methods from disciplines such as history of concepts, linguistics, natural language processing (NLP) and Semantic Web, to create a comparative framework for detecting semantic change in multilingual historical corpora and generating diachronic ontologies as linguistic linked open data (LLOD). Initiated as a use case (UC4.2.1) within the COST Action Nexus Linguarum, European network for Web-centred linguistic data science, the study will explore emerging trends in knowledge extraction, analysis and representation from linguistic data science, and apply the devised methodology to datasets in the humanities to trace the evolution of concepts from the domain of socio-cultural transformation. The paper will describe the main elements of the methodological framework and preliminary planning of the intended workflow."}115{"work_id": 4200503948, "title": "Comparative Analysis of Defect Detection Using YOLO of Deep Learning", "abstract": "There are a huge number of features which are said to improve Convolutional Neural Network (CNN) accuracy.Practical testing of combinations of such features on large datasets, and theoretical justification of the result, is required.Some features operate on certain models exclusively and for certain problems exclusively, or only for small-scale datasets; while some features, such as batch-normalization and residual-connections, are applicable to the majority of models, tasks, and datasets.We assume that such universal features include Weighted-Residual-Connections (WRC), Cross-Stage-Partial-connections (CSP), Cross mini-Batch Normalization (CmBN), Self-adversarial-training (SAT) and Mish-activation.We use new features: WRC, CSP, CmBN, SAT, Mish activation, Mosaic data augmentation, CmBN, DropBlock regularization, and CIoU loss, and combine some of them to achieve state-of-the-art results: 43.5% AP (65.7% AP 50 ) for the MS COCO dataset at a realtime speed of \u223c65 FPS on Tesla V100."}116{"work_id": 4404782964, "title": "A Survey on In-context Learning", "abstract": "Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, Xu Sun, Lei Li, Zhifang Sui. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024."}117{"work_id": 2948223045, "title": "Energy and Policy Considerations for Deep Learning in NLP", "abstract": "Recent progress in hardware and methodology for training neural networks has ushered in a new generation of large networks trained on abundant data.These models have obtained notable gains in accuracy across many NLP tasks.However, these accuracy improvements depend on the availability of exceptionally large computational resources that necessitate similarly substantial energy consumption.As a result these models are costly to train and develop, both financially, due to the cost of hardware and electricity or cloud compute time, and environmentally, due to the carbon footprint required to fuel modern tensor processing hardware.In this paper we bring this issue to the attention of NLP researchers by quantifying the approximate financial and environmental costs of training a variety of recently successful neural network models for NLP.Based on these findings, we propose actionable recommendations to reduce costs and improve equity in NLP research and practice."}118{"work_id": 3196974791, "title": "AST: Audio Spectrogram Transformer", "abstract": "In the past decade, convolutional neural networks (CNNs) have been widely adopted as the main building block for endto-end audio classification models, which aim to learn a direct mapping from audio spectrograms to corresponding labels.To better capture long-range global context, a recent trend is to add a self-attention mechanism on top of the CNN, forming a CNN-attention hybrid model.However, it is unclear whether the reliance on a CNN is necessary, and if neural networks purely based on attention are sufficient to obtain good performance in audio classification.In this paper, we answer the question by introducing the Audio Spectrogram Transformer (AST), the first convolution-free, purely attention-based model for audio classification.We evaluate AST on various audio classification benchmarks, where it achieves new state-of-the-art results of 0.485 mAP on AudioSet, 95.6% accuracy on ESC-50, and 98.1% accuracy on Speech Commands V2."}119{"work_id": 3034971973, "title": "EfficientDet: Scalable and Efficient Object Detection", "abstract": "Model efficiency has become increasingly important in computer vision. In this paper, we systematically study neural network architecture design choices for object detection and propose several key optimizations to improve efficiency. First, we propose a weighted bi-directional feature pyramid network (BiFPN), which allows easy and fast multi-scale feature fusion; Second, we propose a compound scaling method that uniformly scales the resolution, depth, and width for all backbone, feature network, and box/class prediction networks at the same time. Based on these optimizations and EfficientNet backbones, we have developed a new family of object detectors, called EfficientDet, which consistently achieve much better efficiency than prior art across a wide spectrum of resource constraints. In particular, with single-model and single-scale, our EfficientDet-D7 achieves state-of-the-art 52.2 AP on COCO test-dev with 52M parameters and 325B FLOPs1, being 4x - 9x smaller and using 13x - 42x fewer FLOPs than previous detector. Code is available at https://github.com/google/ automl/tree/master/efficientdet."}120{"work_id": 2808631503, "title": "VoxCeleb2: Deep Speaker Recognition", "abstract": "The objective of this paper is speaker recognition under noisy and unconstrained conditions. We make two key contributions. First, we introduce a very large-scale audio-visual speaker recognition dataset collected from open-source media. Using a fully automated pipeline, we curate VoxCeleb2 which contains over a million utterances from over 6,000 speakers. This is several times larger than any publicly available speaker recognition dataset. Second, we develop and compare Convolutional Neural Network (CNN) models and training strategies that can effectively recognise identities from voice under various conditions. The models trained on the VoxCeleb2 dataset surpass the performance of previous works on a benchmark dataset by a significant margin."}121{"work_id": 2990747716, "title": "Learning dexterous in-hand manipulation", "abstract": "We use reinforcement learning (RL) to learn dexterous in-hand manipulation policies that can perform vision-based object reorientation on a physical Shadow Dexterous Hand. The training is performed in a simulated environment in which we randomize many of the physical properties of the system such as friction coefficients and an object\u2019s appearance. Our policies transfer to the physical robot despite being trained entirely in simulation. Our method does not rely on any human demonstrations, but many behaviors found in human manipulation emerge naturally, including finger gaiting, multi-finger coordination, and the controlled use of gravity. Our results were obtained using the same distributed RL system that was used to train OpenAI Five. We also include a video of our results: https://youtu.be/jwSbzNHGflM ."}122{"work_id": 4312443924, "title": "A ConvNet for the 2020s", "abstract": "The \u201cRoaring 20s\u201d of visual recognition began with the introduction of Vision Transformers (ViTs), which quickly superseded ConvNets as the state-of-the-art image classification model. A vanilla ViT, on the other hand, faces difficulties when applied to general computer vision tasks such as object detection and semantic segmentation. It is the hierarchical Transformers (e.g., Swin Transformers) that reintroduced several ConvNet priors, making Transformers practically viable as a generic vision backbone and demonstrating remarkable performance on a wide variety of vision tasks. However, the effectiveness of such hybrid approaches is still largely credited to the intrinsic superiority of Transformers, rather than the inherent inductive biases of convolutions. In this work, we reexamine the design spaces and test the limits of what a pure ConvNet can achieve. We gradually \u201cmodernize\u201d a standard ResNet toward the design of a vision Transformer, and discover several key components that contribute to the performance difference along the way. The outcome of this exploration is a family of pure ConvNet models dubbed ConvNeXt. Constructed entirely from standard ConvNet modules, ConvNeXts compete favorably with Transformers in terms of accuracy and scalability, achieving 87.8% ImageNet top-1 accuracy and outperforming Swin Transformers on COCO detection and ADE20K segmentation, while maintaining the simplicity and efficiency of standard ConvNets."}123{"work_id": 2962730651, "title": "OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields", "abstract": "Realtime multi-person 2D pose estimation is a key component in enabling machines to have an understanding of people in images and videos. In this work, we present a realtime approach to detect the 2D pose of multiple people in an image. The proposed method uses a nonparametric representation, which we refer to as Part Affinity Fields (PAFs), to learn to associate body parts with individuals in the image. This bottom-up system achieves high accuracy and realtime performance, regardless of the number of people in the image. In previous work, PAFs and body part location estimation were refined simultaneously across training stages. We demonstrate that a PAF-only refinement rather than both PAF and body part location refinement results in a substantial increase in both runtime performance and accuracy. We also present the first combined body and foot keypoint detector, based on an internal annotated foot dataset that we have publicly released. We show that the combined detector not only reduces the inference time compared to running them sequentially, but also maintains the accuracy of each component individually. This work has culminated in the release of OpenPose, the first open-source realtime system for multi-person 2D pose detection, including body, foot, hand, and facial keypoints."}124{"work_id": 4286221734, "title": "Human-computer interaction", "abstract": "This study aims to understand the potential of evolving technology to spark inspiration and provide a solution to creative blocks for artists and acting as a catalyst for designers in the ideation stage of design. Specifically, this project evaluates a method delivered through a prototype designed to provide artists with intelligent choices from augmented visual data. Ambiguity and abstraction were carefully used as techniques to initiate subconscious ideation in response to visual media, opening the door to the realm of creativity. This study conducts two rounds of interviews to design and evaluate a method that uses these visual techniques. The data collected from these two rounds of interviews indicates the Squiggle tool effectively inspired the creation of new ideas and assisted in reducing creative blocks by presenting users with abstract visual design choices. This indicates a promising potential with this method, which is to strategically utilize ambiguity and abstraction, like the prototype instanced in this study, to remedy creative blocks and inspire creativity for artists and designers."}125{"work_id": 3119786062, "title": "An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale", "abstract": "While the Transformer architecture has become the de-facto standard for natural language processing tasks, its applications to computer vision remain limited. In vision, attention is either applied in conjunction with convolutional networks, or used to replace certain components of convolutional networks while keeping their overall structure in place. We show that this reliance on CNNs is not necessary and a pure transformer applied directly to sequences of image patches can perform very well on image classification tasks. When pre-trained on large amounts of data and transferred to multiple mid-sized or small image recognition benchmarks (ImageNet, CIFAR-100, VTAB, etc.), Vision Transformer (ViT) attains excellent results compared to state-of-the-art convolutional networks while requiring substantially fewer computational resources to train."}126{"work_id": 4406417959, "title": "Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers", "abstract": "We introduce a language modeling approach for text to speech synthesis (TTS). Specifically, we train aneural codec language model(calledVALL-E) using discrete codes derived from an off-the-shelf neural audio codec model, and regard TTS as a conditional language modeling task rather than continuous signal regression as in previous work. During the pre-training stage, we scale up the TTS training data to 50 k hours of English speech which is hundreds of times larger than existing systems.VALL-Eemergesin-context learningcapability and can be used to synthesize high-quality personalized speech with only a 3-second enrolled recording of an unseen speaker as a prompt. Experiment results show thatVALL-Esignificantly outperforms the state-of-the-art zero-shot TTS system in terms of speech naturalness and speaker similarity. In addition, we findVALL-Ecould preserve the speaker's emotion and acoustic environment from the prompt in synthesis."}127{"work_id": 4200150166, "title": "NeRF", "abstract": "We present a method that achieves state-of-the-art results for synthesizing novel views of complex scenes by optimizing an underlying continuous volumetric scene function using a sparse set of input views. Our algorithm represents a scene using a fully connected (nonconvolutional) deep network, whose input is a single continuous 5D coordinate (spatial location ( x , y , z ) and viewing direction ( \u03b8, \u03d5 )) and whose output is the volume density and view-dependent emitted radiance at that spatial location. We synthesize views by querying 5D coordinates along camera rays and use classic volume rendering techniques to project the output colors and densities into an image. Because volume rendering is naturally differentiable, the only input required to optimize our representation is a set of images with known camera poses. We describe how to effectively optimize neural radiance fields to render photorealistic novel views of scenes with complicated geometry and appearance, and demonstrate results that outperform prior work on neural rendering and view synthesis."}128{"work_id": 4398796510, "title": "A Survey on Vision--Language--Action Models for Embodied AI", "abstract": "Embodied AI is widely recognized as a cornerstone of artificial general intelligence (AGI) because it involves controlling embodied agents to perform tasks in the physical world. Building on the success of large language models (LLMs) and vision-language models (VLMs), a new category of multimodal models-referred to as vision-language-action (VLA) models-has emerged to address language-conditioned robotic tasks in embodied AI by leveraging their distinct ability to generate actions. The recent proliferation of VLAs necessitates a comprehensive survey to capture the rapidly evolving landscape. To this end, we present the first survey on VLAs for embodied AI. This work provides a detailed taxonomy of VLAs, organized into three major lines of research. The first line focuses on individual components of VLAs. The second line is dedicated to developing VLA-based control policies adept at predicting low-level actions. The third line comprises high-level task planners capable of decomposing long-horizon tasks into a sequence of subtasks, thereby guiding VLAs to follow more general user instructions. Furthermore, we provide an extensive summary of relevant resources, including datasets, simulators, and benchmarks. Finally, we discuss the challenges facing VLAs and outline promising future directions in embodied AI. A curated repository associated with this survey is available at: https://github.com/yueen-ma/Awesome-VLA."}129{"work_id": 2986023562, "title": "Mitsuba 2", "abstract": "Modern rendering systems are confronted with a dauntingly large and growing set of requirements: in their pursuit of realism, physically based techniques must increasingly account for intricate properties of light, such as its spectral composition or polarization. To reduce prohibitive rendering times, vectorized renderers exploit coherence via instruction-level parallelism on CPUs and GPUs. Differentiable rendering algorithms propagate derivatives through a simulation to optimize an objective function, e.g., to reconstruct a scene from reference images. Catering to such diverse use cases is challenging and has led to numerous purpose-built systems---partly, because retrofitting features of this complexity onto an existing renderer involves an error-prone and infeasibly intrusive transformation of elementary data structures, interfaces between components, and their implementations (in other words, everything). We propose Mitsuba 2, a versatile renderer that is intrinsically retargetable to various applications including the ones listed above. Mitsuba 2 is implemented in modern C++ and leverages template metaprogramming to replace types and instrument the control flow of components such as BSDFs, volumes, emitters, and rendering algorithms. At compile time, it automatically transforms arithmetic, data structures, and function dispatch, turning generic algorithms into a variety of efficient implementations without the tedium of manual redesign. Possible transformations include changing the representation of color, generating a \"wide\" renderer that operates on bundles of light paths, just-in-time compilation to create computational kernels that run on the GPU, and forward/reverse-mode automatic differentiation. Transformations can be chained, which further enriches the space of algorithms derived from a single generic implementation. We demonstrate the effectiveness and simplicity of our approach on several applications that would be very challenging to create without assistance: a rendering algorithm based on coherent MCMC exploration, a caustic design method for gradient-index optics, and a technique for reconstructing heterogeneous media in the presence of multiple scattering."}130{"work_id": 3102008235, "title": "Addressing Injustices through MOOCs: A study among peri-urban, marginalised, South African youth", "abstract": "The legacies of colonial rule continue to impact everyday life, particularly in education. These structural inequalities are often reinforced and amplified in online \u2018global\u2019 education through a form of digital neocolonialism, which is where hegemonic powers indirectly control or influence marginalised groups through the internet or information technology. In striving for justice-oriented online education models, this study analyses to what extent Massive Open Online Courses (MOOCs), produced both internationally and locally, support (or could support) the needs, preferences, and aspirations of marginalised South African youth and address the material, cultural-epistemic, political, and geopolitical injustices they face. To evaluate what South African peri-urban youth desire in their education and futures, as well as the challenges they experience, a seven-part survey was conducted with 250 youth from five townships in South Africa. Responses showed that whilst participants strongly value and aspire to further their education, financial difficulties, infrastructural barriers, family problems, and lack of emotional support and life mentorship limit them from achieving this. Participants reflected on how colonial and apartheid legacies have affected their educational experiences and identities through inferior quality of education, forced languages, forgotten histories and incongruent values, cultural norms and practices. In parallel, semi-structured interviews were conducted with 35 MOOC designers, from South Africa and the USA, to investigate the ways in which efforts, if any, were being made to reach students most in need of quality education. Interviews covered themes of openness, accessibility, and justice. It was found that, depending on the MOOC designer\u2019s understanding of social justice and decolonial thought, they placed varying emphasis on addressing different forms of injustice. Some focused on resource, access and infrastructural barriers, while others focused on issues of content relevance and knowledge production. Furthermore, MOOC designers\u2019 attempts to address injustices strongly related to their own identities and lived experiences, highlighting the importance of plurality of thought and epistemic diversity in the producers of MOOCs. Drawing on the historical injustices and lived experiences of the youth, and the attempts to address injustices by the MOOC designers, it was ascertained that there is no one size-fits-all formula to creating equitable MOOCs. Rather, depending on the purpose and target audience of the MOOC, nuanced approaches to addressing injustices are suggested. These approaches are shaped by various leverage points that influence the types of, and the extent to which, participatory methods, accessibility measures, knowledge sources, assessment and critical pedagogy are implemented. Additionally, the importance of these leverage points varies over the MOOC\u2019s lifecycle, from inception and design, to implementation and assessment. Bearing in mind the broad-ranging injustices that youth participants raised, these approaches are presented with great caution that educational technologies and open education are not panaceas but if designed and used appropriately and justly, can be tools for liberation."}131{"work_id": 1581527321, "title": "Collision Detection and Avoidance in Computer Controlled Manipulators", "abstract": "This dissertation tackles the problem of planning safe trajectories for computer controlled manipulators with two links and multiple degrees of freedom. There are two ways to look at safe trajectory planning. The first concerns itself with planning trajectories in empty space; obstacles enter into consideration only indirectly in that they determine what part of the maneuverable space is free. The second considers obstacles alone; free space considerations are of secondary importance. We show how these complementary views can be used to advantage in the safe trajectory planning problem. Obstacles are naturally described in cartesian space and trajectories in joint space. If obstacles and trajectories are both represented in one space, collision checks would not require the constant and expensive conversion between the two spaces. We show how it is possible to decompose the planning task so as to get the best of both cartesian space and joint space representations, and yet avoid the constant conversion overhead problem. We show how the principles of hierarchical decomposition can be used to reduce the complexity of the manipulator trajectory planning problem. Different strategies are used for maneuvering far away from obstacles and for maneuvering close to obstacles. A characterization of large chunks of empty space makes maneuvering far away from obstacles very easy. A characterization of obstacle configuration types simplifies planning of maneuvers close to obstacles. The key ideas in the representation that make it possible to realize the above claims are: 1) the identification of a hierarchy of abstraction spaces that permit simplified manipulator descriptions. These spaces make it possible to model the manipulator as two line segments, a single line segment, or incredibly as a point. 2) the identification of primitive trajectory types that make collision detection, trajectory hypothesis and modification numerically tractable. 3) the polyhedra-model of obstacles and the identification of one-time-only transformations on obstacles that significantly simplify trajectory planning. 4) a neat characterization of empty space. Empty space is approximated by easily describable entities called charts; the approximation is dynamic and under program control; the approximation can be selective, and thus it is easy to make incremental modifications to the charts. The thesis describes a model for collision detection and avoidance systems for computer controlled manipulators. The justification for the model lies in the computer implementations for 2D and 3D manipulator systems. These systems incorporate a significant portion of the model. The promising performance of the implementation makes fast collision avoiders a distinct possibility. The solution presented treats manipulators with a sliding joint, and permits the manipulator to transport objects which can be enclosed within the minimum bounding cylinder of the manipulator link. Modifications of the solution that permit handling of large objects are indicated. An extension of the solution that solves the problem for manipulators with only rotary joints is described. A consequence of the investigations into the collision detection and avoidance problem has been the identification of execution-time strategies for terminal phase motion. Guidelines have been presented for incorporating proximity sensors into the manipulator system."}132{"work_id": 4392239564, "title": "Human-AI collaboration patterns in AI-assisted academic writing", "abstract": "Artificial Intelligence (AI) has increasingly influenced higher education, notably in academic writing where AI-powered assisting tools offer both opportunities and challenges.Recently, the rapid growth of generative AI (GAI) has brought its impacts into sharper focus, yet the dynamics of its utilisation in academic writing remain largely unexplored.This paper focuses on examining the nature of human-AI interactions in academic writing, specifically investigating the strategies doctoral students employ when collaborating with a GAI-powered assisting tool.This study involves 626 recorded activities on how ten doctoral students interact with GAI-powered assisting tool during academic writing.AIdriven learning analytics approach was adopted for three layered analyses: (1) data pre-processing and analysis with quantitative content analysis, (2) sequence analysis with Hidden Markov Model (HMM) and hierarchical sequence clustering, and (3) pattern analysis with process mining.Findings indicate that doctoral students engaging in iterative, highly interactive processes with the GAI-powered assisting tool generally achieve better performance in the writing task.In contrast, those who use GAI merely as a supplementary information source, maintaining a linear writing approach, tend to get lower writing performance.This study points to the need for further investigations into human-AI collaboration in learning in higher education, with implications for tailored educational strategies and solutions."}133{"work_id": 2890964092, "title": "X-Vectors: Robust DNN Embeddings for Speaker Recognition", "abstract": "In this paper, we use data augmentation to improve performance of deep neural network (DNN) embeddings for speaker recognition. The DNN, which is trained to discriminate between speakers, maps variable-length utterances to fixed-dimensional embeddings that we call x-vectors. Prior studies have found that embeddings leverage large-scale training datasets better than i-vectors. However, it can be challenging to collect substantial quantities of labeled data for training. We use data augmentation, consisting of added noise and reverberation, as an inexpensive method to multiply the amount of training data and improve robustness. The x-vectors are compared with i-vector baselines on Speakers in the Wild and NIST SRE 2016 Cantonese. We find that while augmentation is beneficial in the PLDA classifier, it is not helpful in the i-vector extractor. However, the x-vector DNN effectively exploits data augmentation, due to its supervised training. As a result, the x-vectors achieve superior performance on the evaluation datasets."}134{"work_id": 4399978538, "title": "Exploring Augmented Reality Integration in Diagnostic Imaging: Myth or Reality?", "abstract": "This study delves into the transformative potential of integrating augmented reality (AR) within imaging technologies, shedding light on this evolving landscape. Through a comprehensive narrative review, this research uncovers a wealth of literature exploring the intersection between AR and medical imaging, highlighting its growing prominence in healthcare. AR's integration offers a host of potential opportunities to enhance surgical precision, bolster patient engagement, and customize medical interventions. Moreover, when combined with technologies like virtual reality (VR), artificial intelligence (AI), and robotics, AR opens up new avenues for innovation in clinical practice, education, and training. However, amidst these promising prospects lie numerous unanswered questions and areas ripe for exploration. This study emphasizes the need for rigorous research to elucidate the clinical efficacy of AR-integrated interventions, optimize surgical workflows, and address technological challenges. As the healthcare landscape continues to evolve, sustained research efforts are crucial to fully realizing AR's transformative impact in medical imaging. Systematic reviews on AR in healthcare also overlook regulatory and developmental factors, particularly in regard to medical devices. These include compliance with standards, safety regulations, risk management, clinical validation, and developmental processes. Addressing these aspects will provide a comprehensive understanding of the challenges and opportunities in integrating AR into clinical settings, informing stakeholders about crucial regulatory and developmental considerations for successful implementation. Moreover, navigating the regulatory approval process requires substantial financial resources and expertise, presenting barriers to entry for smaller innovators. Collaboration across disciplines and concerted efforts to overcome barriers will be essential in navigating this frontier and harnessing the potential of AR to revolutionize healthcare delivery."}135{"work_id": 2975281968, "title": "Modeling Music : Studies of Music Transcription, Music Perception and Music Production", "abstract": "This dissertation presents ten studies focusing on three important subfields of music information retrieval (MIR): music transcription (Part A), music perception (Part B), and music production (Part C). In Part A, systems capable of transcribing rhythm and polyphonic pitch are described. The first two publications present methods for tempo estimation and beat tracking. A method is developed for computing the most salient periodicity (the \u201ccepstroid\u201d), and the computed cepstroid is used to guide the machine learning processing. The polyphonic pitch tracking system uses novel pitch-invariant and tone-shift-invariant processing techniques. Furthermore, the neural flux is introduced \u2013 a latent feature for onset and offset detection. The transcription systems use a layered learning technique with separate intermediate networks of varying depth. Important music concepts are used as intermediate targets to create a processing chain with high generalization. State-of-the-art performance is reported for all tasks. Part B is devoted to perceptual features of music, which can be used as intermediate targets or as parameters for exploring fundamental music perception mechanisms. Systems are proposed that can predict the perceived speed and performed dynamics of an audio file with high accuracy, using the average ratings from around 20 listeners as ground truths. In Part C, aspects related to music production are explored. The first paper analyzes long-term average spectrum (LTAS) in popular music. A compact equation is derived to describe the mean LTAS of a large dataset, and the variation is visualized. Further analysis shows that the level of the percussion is an important factor for LTAS. The second paper examines songwriting and composition through the development of an algorithmic composer of popular music. Various factors relevant for writing good compositions are encoded, and a listening test employed that shows the validity of the proposed methods. The dissertation is concluded by Part D - Looking Back and Ahead, which acts as a discussion and provides a road-map for future work. The first paper discusses the deep layered learning (DLL) technique, outlining concepts and pointing out a direction for future MIR implementations. It is suggested that DLL can help generalization by enforcing the validity of intermediate representations, and by letting the inferred representations establish disentangled structures supporting high-level invariant processing. The second paper proposes an architecture for tempo-invariant processing of rhythm with convolutional neural networks. Log-frequency representations of rhythm-related activations are suggested at the main stage of processing. Methods relying on magnitude, relative phase, and raw phase information are described for a wide variety of rhythm processing tasks."}136{"work_id": 2979736514, "title": "Neural transfer learning for natural language processing", "abstract": "The current generation of neural network-based natural language processing models excels at learning from large amounts of labelled data. Given these capabilities, natural language processing is increasingly applied to new tasks, new domains, and new languages. Current models, however, are sensitive to noise and adversarial examples and prone to overfitting. This brittleness, together with the cost of attention, challenges the supervised learning paradigm. Transfer learning allows us to leverage knowledge acquired from related data in order to improve performance on a target task. Implicit transfer learning in the form of pretrained word representations has been a common component in natural language processing. In this dissertation, we argue that more explicit transfer learning is key to deal with the dearth of training data and to improve downstream performance of natural language processing models. We show experimental results transferring knowledge from related domains, tasks, and languages that support this hypothesis. We make several contributions to transfer learning for natural language processing: Firstly, we propose new methods to automatically select relevant data for supervised and unsupervised domain adaptation. Secondly, we propose two novel architectures that improve sharing in multi-task learning and outperform single-task learning as well as the state-of-the-art. Thirdly, we analyze the limitations of current models for unsupervised cross-lingual transfer and propose a method to mitigate them as well as a novel latent variable cross-lingual word embedding model. Finally, we propose a framework based on fine-tuning language models for sequential transfer learning and analyze the adaptation phase."}137{"work_id": 2955058313, "title": "Dual Attention Network for Scene Segmentation", "abstract": "In this paper, we address the scene segmentation task by capturing rich contextual dependencies based on the self-attention mechanism. Unlike previous works that capture contexts by multi-scale features fusion, we propose a Dual Attention Networks (DANet) to adaptively integrate local features with their global dependencies. Specifically, we append two types of attention modules on top of traditional dilated FCN, which model the semantic interdependencies in spatial and channel dimensions respectively. The position attention module selectively aggregates the features at each position by a weighted sum of the features at all positions. Similar features would be related to each other regardless of their distances. Meanwhile, the channel attention module selectively emphasizes interdependent channel maps by integrating associated features among all channel maps. We sum the outputs of the two attention modules to further improve feature representation which contributes to more precise segmentation results. We achieve new state-of-the-art segmentation performance on three challenging scene segmentation datasets, i.e., Cityscapes, PASCAL Context and COCO Stuff dataset. In particular, a Mean IoU score of 81.5% on Cityscapes test set is achieved without using coarse data."}138{"work_id": 2980282514, "title": "HuggingFace's Transformers: State-of-the-art Natural Language Processing", "abstract": "Recent progress in natural language processing has been driven by advances in both model architecture and model pretraining. Transformer architectures have facilitated building higher-capacity models and pretraining has made it possible to effectively utilize this capacity for a wide variety of tasks. \\textit{Transformers} is an open-source library with the goal of opening up these advances to the wider machine learning community. The library consists of carefully engineered state-of-the art Transformer architectures under a unified API. Backing this library is a curated collection of pretrained models made by and available for the community. \\textit{Transformers} is designed to be extensible by researchers, simple for practitioners, and fast and robust in industrial deployments. The library is available at \\url{https://github.com/huggingface/transformers}."}139{"work_id": 4386072096, "title": "DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation", "abstract": "Large text-to-image models achieved a remarkable leap in the evolution of AI, enabling high-quality and diverse synthesis of images from a given text prompt. However, these models lack the ability to mimic the appearance of subjects in a given reference set and synthesize novel renditions of them in different contexts. In this work, we present a new approach for \u201cpersonalization\u201d of text-to-image diffusion models. Given as input just a few images of a subject, we fine-tune a pretrained text-to-image model such that it learns to bind a unique identifier with that specific subject. Once the subject is embedded in the output domain of the model, the unique identifier can be used to synthesize novel photorealistic images of the subject contextualized in different scenes. By leveraging the semantic prior embedded in the model with a new autogenous class-specific prior preservation loss, our technique enables synthesizing the subject in diverse scenes, poses, views and lighting conditions that do not appear in the reference images. We apply our technique to several previously-unassailable tasks, including subject recontextualization, text-guided view synthesis, and artistic rendering, all while preserving the subject's key features. We also provide a new dataset and evaluation protocol for this new task of subject-driven generation. Project page: https://dreambooth.github.io/"}140{"work_id": 4387835442, "title": "Generative Agents: Interactive Simulacra of Human Behavior", "abstract": "Believable proxies of human behavior can empower interactive applications ranging from immersive environments to rehearsal spaces for interpersonal communication to prototyping tools. In this paper, we introduce generative agents: computational software agents that simulate believable human behavior. Generative agents wake up, cook breakfast, and head to work; artists paint, while authors write; they form opinions, notice each other, and initiate conversations; they remember and reflect on days past as they plan the next day. To enable generative agents, we describe an architecture that extends a large language model to store a complete record of the agent\u2019s experiences using natural language, synthesize those memories over time into higher-level reflections, and retrieve them dynamically to plan behavior. We instantiate generative agents to populate an interactive sandbox environment inspired by The Sims, where end users can interact with a small town of twenty-five agents using natural language. In an evaluation, these generative agents produce believable individual and emergent social behaviors. For example, starting with only a single user-specified notion that one agent wants to throw a Valentine\u2019s Day party, the agents autonomously spread invitations to the party over the next two days, make new acquaintances, ask each other out on dates to the party, and coordinate to show up for the party together at the right time. We demonstrate through ablation that the components of our agent architecture\u2014observation, planning, and reflection\u2014each contribute critically to the believability of agent behavior. By fusing large language models with computational interactive agents, this work introduces architectural and interaction patterns for enabling believable simulations of human behavior."}141{"work_id": 4225672218, "title": "Restormer: Efficient Transformer for High-Resolution Image Restoration", "abstract": "Since convolutional neural networks (CNNs) perform well at learning generalizable image priors from large-scale data, these models have been extensively applied to image restoration and related tasks. Recently, another class of neural architectures, Transformers, have shown significant performance gains on natural language and high-level vision tasks. While the Transformer model mitigates the shortcomings of CNNs (i.e., limited receptive field and inadaptability to input content), its computational complexity grows quadratically with the spatial resolution, therefore making it infeasible to apply to most image restoration tasks involving high-resolution images. In this work, we propose an efficient Transformer model by making several key designs in the building blocks (multi-head attention and feed-forward network) such that it can capture long-range pixel interactions, while still remaining applicable to large images. Our model, named Restoration Transformer (Restormer), achieves state-of-the-art results on several image restoration tasks, including image deraining, single-image motion deblurring, defocus deblurring (single-image and dual-pixel data), and image denoising (Gaussian grayscale/color denoising, and real image denoising). The source code and pre-trained models are available at https://github.com/swz30/Restormer."}142{"work_id": 7130436101, "title": "An agentic system for rare disease diagnosis with traceable reasoning", "abstract": "Rare diseases affect more than 300 million people worldwide1\u20133, yet timely and accurate diagnosis remains an urgent challenge1,3\u20135. Patients often endure a prolonged \u2018diagnostic odyssey\u2019 exceeding 5 years, marked by repeated referrals, misdiagnoses and unnecessary interventions, leading to delayed treatment and substantial emotional and economic burden4,5. Here we present DeepRare\u2014a multi-agent system for rare disease differential diagnosis decision support6\u20138 powered by large language models, integrating more than 40 specialized tools and up-to-date knowledge sources. DeepRare processes heterogeneous clinical inputs, including free-text descriptions, structured human phenotype ontology terms and genetic testing results to generate ranked diagnostic hypotheses with transparent reasoning linked to verifiable medical evidence. Evaluated across nine datasets from literature, case reports and clinical centres across Asia, North America and Europe spanning 14 medical specialties, DeepRare demonstrates exceptional performance on 2,919 diseases. In human-phenotype-ontology-based tasks, it achieves an average Recall@1 of 57.18%, outperforming the next best method by 23.79%; in multi-modal tests, it reaches 69.1% compared with Exomiser\u2019s 55.9% on 168 cases. Expert review achieved 95.4% agreement on its reasoning chains, confirming their validity and traceability. Our work not only advances rare disease diagnosis but also demonstrates how the latest powerful large-language-model-driven agentic systems can reshape current clinical workflows. DeepRare\u2014a multi-agent system for rare disease differential diagnosis decision support powered by large language models, integrating specialized tools and up-to-date knowledge sources\u2014has the potential to reduce healthcare disparities in rare disease diagnosis."}143{"work_id": 7106270182, "title": "Bridging the Gap between Object and Image-level Representations for Open-Vocabulary Detection", "abstract": "Existing open-vocabulary object detectors typically enlarge their vocabulary sizes by leveraging different forms of weak supervision. This helps generalize to novel objects at inference. Two popular forms of weak-supervision used in open-vocabulary detection (OVD) include pretrained CLIP model and image-level supervision. We note that both these modes of supervision are not optimally aligned for the detection task: CLIP is trained with image-text pairs and lacks precise localization of objects while the image-level supervision has been used with heuristics that do not accurately specify local object regions. In this work, we propose to address this problem by performing object-centric alignment of the language embeddings from the CLIP model. Furthermore, we visually ground the objects with only image-level supervision using a pseudo-labeling process that provides high-quality object proposals and helps expand the vocabulary during training. We establish a bridge between the above two object-alignment strategies via a novel weight transfer function that aggregates their complimentary strengths. In essence, the proposed model seeks to minimize the gap between object and image-centric representations in the OVD setting. On the COCO benchmark, our proposed approach achieves 36.6 AP-50 on novel classes, an absolute 8.2 gain over the previous best performance. For LVIS, we surpass the state-of-the-art ViLD model by 5.0 mask AP for rare categories and 3.4 overall."}144{"work_id": 3163641770, "title": "Metric Type in the Target-matrix Mesh Optimization Paradigm", "abstract": "The Target Matrix Optimization Paradigm (TMOP) is a method for improving the accuracy, efficiency, and robustness of numerical solutions to partial differential equations by improving the geometric quality of the computational mesh, primarily through node movement.TMOP has been sucessfully applied to a number applications even though the paradigm was not fully understood at the time.With this work, TMOP can be seen to be a tightly woven fabric of interconnecting ideas and concepts that provides a powerful approach to mesh optimization.The central unifying concepts in TMOP are the concept of a Target Matrix and the concept of Metric Type.Targetmatrices are motivated by the desire to make mesh quality improvement application-specific and, when needed, solution-adaptive.Metric type plays an essential role because it provides, through the use of typed metrics, the bridge between application-specific quality and target construction.It is shown that there are eight theoretical metric types, including the shape and shape+size types used informally in the past.It is shown further that there exist well-posed metrics corresponding to six of the eight metric types.A well-posed metric is a metric that is typed and convex, polyconvex, or invex, and further, it is a metric that simplifies target construction."}145{"work_id": 4390874575, "title": "Segment Anything", "abstract": "We introduce the Segment Anything (SA) project: a new task, model, and dataset for image segmentation. Using our efficient model in a data collection loop, we built the largest segmentation dataset to date (by far), with over 1 billion masks on 11M licensed and privacy respecting images. The model is designed and trained to be promptable, so it can transfer zero-shot to new image distributions and tasks. We evaluate its capabilities on numerous tasks and find that its zero-shot performance is impressive \u2013 often competitive with or even superior to prior fully supervised results. We are releasing the Segment Anything Model (SAM) and corresponding dataset (SA-1B) of 1B masks and 11M images at segment-anything.com to foster research into foundation models for computer vision. We recommend reading the full paper at: arxiv.org/abs/2304.02643."}146{"work_id": 3159481202, "title": "Emerging Properties in Self-Supervised Vision Transformers", "abstract": "In this paper, we question if self-supervised learning provides new properties to Vision Transformer (ViT) [16] that stand out compared to convolutional networks (convnets). Beyond the fact that adapting self-supervised methods to this architecture works particularly well, we make the following observations: first, self-supervised ViT features contain explicit information about the semantic segmentation of an image, which does not emerge as clearly with supervised ViTs, nor with convnets. Second, these features are also excellent k-NN classifiers, reaching 78.3% top-1 on ImageNet with a small ViT. Our study also underlines the importance of momentum encoder [26], multi-crop training [9], and the use of small patches with ViTs. We implement our findings into a simple self-supervised method, called DINO, which we interpret as a form of self-distillation with no labels. We show the synergy between DINO and ViTs by achieving 80.1% top-1 on ImageNet in linear evaluation with ViT-Base."}147{"work_id": 3113347918, "title": "NeRD: Neural Reflectance Decomposition from Image Collections", "abstract": "Decomposing a scene into its shape, reflectance, and illumination is a challenging but important problem in computer vision and graphics. This problem is inherently more challenging when the illumination is not a single light source under laboratory conditions but is instead an unconstrained environmental illumination. Though recent work has shown that implicit representations can be used to model the radiance field of an object, most of these techniques only enable view synthesis and not relighting. Additionally, evaluating these radiance fields is resource and time-intensive. We propose a neural reflectance decomposition (NeRD) technique that uses physically-based rendering to decompose the scene into spatially varying BRDF material properties. In contrast to existing techniques, our input images can be captured under different illumination conditions. In addition, we also propose techniques to convert the learned reflectance volume into a relightable textured mesh enabling fast real-time rendering with novel illuminations. We demonstrate the potential of the proposed approach with experiments on both synthetic and real datasets, where we are able to obtain high-quality relightable 3D assets from image collections. The datasets and code are available at the project page: https://markboss.me/publication/2021-nerd/."}148{"work_id": 3182555022, "title": "Generative Adversarial Networks for Semantic Image Synthesis and Unconditional Synthesis with Limited Data", "abstract": "Generative modeling of images is an important task aimed at synthesizing new images that are indistinguishable from real samples. The ability to generate diverse, realistic- looking images holds numerous applications, ranging from artistic content creation to data augmentation for other computer vision tasks. In this thesis, we study generative adversarial networks (GANs), which have gained popularity as the leading image synthesis framework due to their exceptional performance. GANs consist of a generator and a discriminator that are engaged in an adversarial game. In this game, the discriminator learns to correctly classify incoming real and fake images, while the generator is trained to fool the discriminator by producing more realistic samples. Although GANs have made significant advancements in recent years, more research is needed to further enhance their scalability across diverse datasets, as well as the quality and diversity of their synthesis. To this end, this thesis presents several new GAN methods that expand the literature on GANs in various aspects. \\n \\nFirstly, we address the task of semantic image synthesis, generating realistic images from semantic label maps. For this task, our work introduces a new segmentation-based discriminator that provides a strong training signal for the generator, eliminating the need for additional losses and tricks used in prior models. Compared to previous GAN approaches, our proposed method achieves a synthesis with higher image quality and diversity, while showing much better scalability to datasets with severe class imbalances. \\n \\nSecondly, we explore the training of unconditional GAN models in low data regimes. Previously, under limited data, GAN models suffered from training instabilities and memorization issues, which limited their application in restricted image domains. To address this, we propose new GAN approaches for various limited-data scenarios, including traditional one-shot and few-shot learning regimes. The advancements of our methods include new training schemes, improved architectures of discriminators, and novel regularization terms for generators. Our proposed methods enable high-quality and diverse synthesis from extremely small datasets, on which prior GAN models could not be trained successfully. \\n \\nOverall, this thesis advances the field of generative adversarial networks by introducing new GAN models that improve over prior work in the image synthesis quality, diversity, and applicability across various image domains. The proposed approaches demonstrate superior performance in semantic image synthesis and unconditional training with limited data, making GANs more powerful and effective for a wide range of computer vision applications."}149{"work_id": 2979750740, "title": "Dynamic Graph CNN for Learning on Point Clouds", "abstract": "Point clouds provide a flexible geometric representation suitable for countless applications in computer graphics; they also comprise the raw output of most 3D data acquisition devices. While hand-designed features on point clouds have long been proposed in graphics and vision, however, the recent overwhelming success of convolutional neural networks (CNNs) for image analysis suggests the value of adapting insight from CNN to the point cloud world. Point clouds inherently lack topological information, so designing a model to recover topology can enrich the representation power of point clouds. To this end, we propose a new neural network module dubbed EdgeConv suitable for CNN-based high-level tasks on point clouds, including classification and segmentation. EdgeConv acts on graphs dynamically computed in each layer of the network. It is differentiable and can be plugged into existing architectures. Compared to existing modules operating in extrinsic space or treating each point independently, EdgeConv has several appealing properties: It incorporates local neighborhood information; it can be stacked applied to learn global shape properties; and in multi-layer systems affinity in feature space captures semantic characteristics over potentially long distances in the original embedding. We show the performance of our model on standard benchmarks, including ModelNet40, ShapeNetPart, and S3DIS."}150{"work_id": 4300424419, "title": "U-Prithvi: Integrating a Foundation Model and U-Net for Enhanced Flood Inundation Mapping", "abstract": "In recent years, large pre-trained models, commonly referred to as foundation models, have become increasingly popular for various tasks leveraging transfer learning. This trend has expanded to remote sensing, where transformer-based foundation models such as Prithvi, msGFM, and SatSwinMAE have been utilized for a range of applications. While these transformer-based models, particularly the Prithvi model, exhibit strong generalization capabilities, they have limitations on capturing fine-grained details compared to convolutional neural network architectures like U-Net in segmentation tasks. In this paper, we propose a novel architecture, U-Prithvi, which combines the strengths of the Prithvi transformer with those of U-Net. We introduce a RandomHalfMaskLayer to ensure balanced learning from both models during training. Our approach is evaluated on the Sen1Floods11 dataset for flood inundation mapping, and experimental results demonstrate better performance of U-Prithvi over both individual models, achieving improved performance on out-of-sample data. While this principle is illustrated using the Prithvi model, it is easily adaptable to other foundation models."}151{"work_id": 4385473486, "title": "RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control", "abstract": "We study how vision-language models trained on Internet-scale data can be incorporated directly into end-to-end robotic control to boost generalization and enable emergent semantic reasoning. Our goal is to enable a single end-to-end trained model to both learn to map robot observations to actions and enjoy the benefits of large-scale pretraining on language and vision-language data from the web. To this end, we propose to co-fine-tune state-of-the-art vision-language models on both robotic trajectory data and Internet-scale vision-language tasks, such as visual question answering. In contrast to other approaches, we propose a simple, general recipe to achieve this goal: in order to fit both natural language responses and robotic actions into the same format, we express the actions as text tokens and incorporate them directly into the training set of the model in the same way as natural language tokens. We refer to such category of models as vision-language-action models (VLA) and instantiate an example of such a model, which we call RT-2. Our extensive evaluation (6k evaluation trials) shows that our approach leads to performant robotic policies and enables RT-2 to obtain a range of emergent capabilities from Internet-scale training. This includes significantly improved generalization to novel objects, the ability to interpret commands not present in the robot training data (such as placing an object onto a particular number or icon), and the ability to perform rudimentary reasoning in response to user commands (such as picking up the smallest or largest object, or the one closest to another object). We further show that incorporating chain of thought reasoning allows RT-2 to perform multi-stage semantic reasoning, for example figuring out which object to pick up for use as an improvised hammer (a rock), or which type of drink is best suited for someone who is tired (an energy drink)."}152{"work_id": 2925148167, "title": "nuScenes: A Multimodal Dataset for Autonomous Driving", "abstract": "Robust detection and tracking of objects is crucial for the deployment of autonomous vehicle technology. Image based benchmark datasets have driven development in computer vision tasks such as object detection, tracking and segmentation of agents in the environment. Most autonomous vehicles, however, carry a combination of cameras and range sensors such as lidar and radar. As machine learning based methods for detection and tracking become more prevalent, there is a need to train and evaluate such methods on datasets containing range sensor data along with images. In this work we present nuTonomy scenes (nuScenes), the first dataset to carry the full autonomous vehicle sensor suite: 6 cameras, 5 radars and 1 lidar, all with full 360 degree field of view. nuScenes comprises 1000 scenes, each 20s long and fully annotated with 3D bounding boxes for 23 classes and 8 attributes. It has 7x as many annotations and 100x as many images as the pioneering KITTI dataset. We define novel 3D detection and tracking metrics. We also provide careful dataset analysis as well as baselines for lidar and image based detection and tracking. Data, development kit and more information are available online."}153{"work_id": 4385430674, "title": "Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware", "abstract": "Meta Fig. 1: ALOHA : A Low-cost Open-source Hardware System for Bimanual Teleoperation.The whole system costs <$20k with off-the-shelf robots and 3D printed components.Left: The user teleoperates by backdriving the leader robots, with the follower robots mirroring the motion.Right: ALOHA is capable of precise, contact-rich, and dynamic tasks.We show examples of both teleoperated and learned skills.Abstract-Fine manipulation tasks, such as threading cable ties or slotting a battery, are notoriously difficult for robots because they require precision, careful coordination of contact forces, and closed-loop visual feedback.Performing these tasks typically requires high-end robots, accurate sensors, or careful calibration, which can be expensive and difficult to set up.Can learning enable low-cost and imprecise hardware to perform these fine manipulation tasks?We present a low-cost system that performs end-to-end imitation learning directly from real demonstrations, collected with a custom teleoperation interface.Imitation learning, however, presents its own challenges, particularly in high-precision domains: errors in the policy can compound over time, and human demonstrations can be non-stationary.To address these challenges, we develop a simple yet novel algorithm, Action Chunking with Transformers (ACT), which learns a generative model over action sequences.ACT allows the robot to learn 6 difficult tasks in the real world, such as opening a translucent condiment cup and slotting a battery with 80-90% success, with only 10 minutes worth of demonstrations.Project website: tonyzhaozh.github.io/aloha"}154{"work_id": 3144501610, "title": "Differentiable visual computing", "abstract": "This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections."}155{"work_id": 4402988539, "title": "Artificial Intelligence-Enabled Intelligent Assistant for Personalized and Adaptive Learning in Higher Education", "abstract": "This paper presents a novel framework, artificial intelligence-enabled intelligent assistant (AIIA), for personalized and adaptive learning in higher education. The AIIA system leverages advanced AI and natural language processing (NLP) techniques to create an interactive and engaging learning platform. This platform is engineered to reduce cognitive load on learners by providing easy access to information, facilitating knowledge assessment, and delivering personalized learning support tailored to individual needs and learning styles. The AIIA\u2019s capabilities include understanding and responding to student inquiries, generating quizzes and flashcards, and offering personalized learning pathways. The research findings have the potential to significantly impact the design, implementation, and evaluation of AI-enabled virtual teaching assistants (VTAs) in higher education, informing the development of innovative educational tools that can enhance student learning outcomes, engagement, and satisfaction. The paper presents the methodology, system architecture, intelligent services, and integration with learning management systems (LMSs) while discussing the challenges, limitations, and future directions for the development of AI-enabled intelligent assistants in education."}156{"work_id": 4224308101, "title": "PaLM: Scaling Language Modeling with Pathways", "abstract": "Large language models have been shown to achieve remarkable performance across a variety of natural language tasks using few-shot learning, which drastically reduces the number of task-specific training examples needed to adapt the model to a particular application. To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model PaLM. We trained PaLM on 6144 TPU v4 chips using Pathways, a new ML system which enables highly efficient training across multiple TPU Pods. We demonstrate continued benefits of scaling by achieving state-of-the-art few-shot learning results on hundreds of language understanding and generation benchmarks. On a number of these tasks, PaLM 540B achieves breakthrough performance, outperforming the finetuned state-of-the-art on a suite of multi-step reasoning tasks, and outperforming average human performance on the recently released BIG-bench benchmark. A significant number of BIG-bench tasks showed discontinuous improvements from model scale, meaning that performance steeply increased as we scaled to our largest model. PaLM also has strong capabilities in multilingual tasks and source code generation, which we demonstrate on a wide array of benchmarks. We additionally provide a comprehensive analysis on bias and toxicity, and study the extent of training data memorization with respect to model scale. Finally, we discuss the ethical considerations related to large language models and discuss potential mitigation strategies."}157{"work_id": 2949117887, "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "abstract": "Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. We refer to this phenomenon as internal covariate shift, and address the problem by normalizing layer inputs. Our method draws its strength from making normalization a part of the model architecture and performing the normalization for each training mini-batch. Batch Normalization allows us to use much higher learning rates and be less careful about initialization. It also acts as a regularizer, in some cases eliminating the need for Dropout. Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin. Using an ensemble of batch-normalized networks, we improve upon the best published result on ImageNet classification: reaching 4.9% top-5 validation error (and 4.8% test error), exceeding the accuracy of human raters."}158{"work_id": 4399300671, "title": "The effects of back\u2010projection variants in BPF\u2010like TOF PET reconstruction using CNN filtration \u2013 Based on simulated and clinical brain data", "abstract": "Abstract Background The back\u2010projection strategies such as confidence weighting (CW) and most likely annihilation position (MLAP) have been adopted into back\u2010projection\u2010and\u2010filtering\u2010like (BPF\u2010like) deep reconstruction model and shown great potential on fast and accurate PET reconstruction. Although the two methods degenerate to an identical model at the time resolution of 0 ps, they represent two distinct approaches at the realistic time resolutions of current commercial systems. There is a lack of a systematic and fair assessment on these differences. Purpose This work aims to analyze the impact of back\u2010projection variants on CNN\u2010based PET image reconstruction to find the most effective back\u2010projection model, and ultimately contribute to accurate PET reconstruction. Methods Different back\u2010projection strategies (CW and MLAP) and different angular view processing methods (view\u2010summed and view\u2010grouped) were considered, leading to the comparison of four back\u2010projection variants integrated with the same CNN filtration model. Meanwhile, we investigated two strategies of physical effect compensation, either introducing pre\u2010corrected data as the input or adding a channel of attenuation map to the CNN model. After training models separately on Monte\u2010Carlo\u2010simulated BrainWeb phantoms with full dose (events = 3\u00d7107), we tested them on both simulated phantoms and clinical brain scans with two dosage levels. For the performance assessment, peak signal\u2010to\u2010noise ratio (PSNR) and root mean square error (RMSE) were used to evaluate the pixel\u2010wise error, structural similarity index (SSIM) to evaluate the structural similarity, and contrast recovery coefficient (CRC) in manually selected ROI to compare the region recovery. Results Compared to two MLAP\u2010based histo\u2010image reconstruction models, two CW\u2010based back\u2010projected image methods produced clearer, sharper, and more detailed images, from both simulated and clinical data. For angular view processing methods, view\u2010grouped histo\u2010image improved image quality, while view\u2010grouped cwbp\u2010image showed no advantage except for contrast recovery. Quantitative analysis on simulated data demonstrated that the view\u2010summed cwbp\u2010image model achieved the best PSNR, RMSE, SSIM, while the 8\u2010view cwbp\u2010image model achieved the best CRC in lesions and the white matter. Additionally, the multi\u2010channel input model including the back\u2010projection image and attenuation map was proved to be the most efficient and simplest method for compensating for physical effects for brain data. Applying Gaussian blur to the histo\u2010image yielded images with limited improvement. All above results hold for both the half\u2010dose and the full\u2010dose cases. Conclusion For brain imaging, the evaluation based on metrics PSNR, RMSE, SSIM, and CRC indicates that the view\u2010summed CW\u2010based back\u2010projection variant is the most effective input for the BPF\u2010like reconstruction model using CNN filtration, which can involve the attenuation map through an additional channel to effectively compensate for physical effects."}159{"work_id": 2979826702, "title": "Transformers: State-of-the-Art Natural Language Processing", "abstract": "Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, Alexander Rush. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. 2020."}160{"work_id": 4402727764, "title": "Improved Baselines with Visual Instruction Tuning", "abstract": "Large multimodal models (LMM) have recently shown encouraging progress with visual instruction tuning. In this paper, we present the first systematic study to investigate the design choices of LMMs in a controlled setting under the LLaVA framework. We show that the fully-connected vision-language connector in LLaVA is surprisingly power-ful and data-efficient. With simple modifications to LLa VA, namely, using CLIP- ViT-L-336px with an MLP projection and adding academic-task-oriented VQA data with response formatting prompts, we establish stronger baselines that achieve state-of-the-art across 11 benchmarks. Our final 13B checkpoint uses merely 1.2M publicly available data, and finishes full training in ~ 1 day on a single 8-AI00 node. Furthermore, we present some early exploration of open problems in LMMs, including scaling to higher resolution inputs, compositional capabilities, and model hallucination, etc. We hope this makes state-of-the-art LMM research more accessible. Code and model will be publicly available."}161{"work_id": 1797268635, "title": "Caltech-UCSD Birds-200-2011 Dataset", "abstract": "CUB-200-2011 is an extended version of CUB-200 [7], a challenging dataset of 200 bird species. The extended version roughly doubles the number of images per category and adds new part localization annotations. All images are annotated with bounding boxes, part locations, and at- tribute labels. Images and annotations were filtered by mul- tiple users of Mechanical Turk. We introduce benchmarks and baseline experiments for multi-class categorization and part localization."}162{"work_id": 1483393136, "title": "Advanced high dynamic range imaging : theory and practice", "abstract": "Imaging techniques seek to simulate the array of light that reaches our eyes to provide the illusion of sensing scenes directly. Both photography and computer graphics deal with the generation of images. Both disciplines have to cope with the high dynamic range in the energy of visible light that human eyes can sense. Traditionally photography and computer graphics took different approaches to the high dynamic range problem. Work over the last ten years though has unified these disciplines and created powerful new tools for the creation of complex, compelling and realistic images. This book provides a practical introduction to the emerging new discipline of high dynamic range imaging that combines photography and computer graphics. By providing detailed equations and code, the book gives the reader the tools needed to experiment with new techniques for creating compelling images. A supplemental website contains downloads and additional information."}163{"work_id": 4287824654, "title": "Question Answering For Toxicological Information Extraction", "abstract": "Working with large amounts of text data has become hectic and time-consuming. In order to reduce human effort, costs, and make the process more efficient, companies and organizations resort to intelligent algorithms to automate and assist the manual work. This problem is also present in the field of toxicological analysis of chemical substances, where information needs to be searched from multiple documents. That said, we propose an approach that relies on Question Answering for acquiring information from unstructured data, in our case, English PDF documents containing information about physicochemical and toxicological properties of chemical substances. Experimental results confirm that our approach achieves promising results which can be applicable in the business scenario, especially if further revised by humans."}164{"work_id": 3118608800, "title": "Learning Multiple Layers of Features from Tiny Images", "abstract": "April 8, 2009Groups at MIT and NYU have collected a dataset of millions of tiny colour images from the web. It is, in principle, an excellent dataset for unsupervised training of deep generative models, but previous researchers who have tried this have found it di cult to learn a good set of lters from the images. We show how to train a multi-layer generative model that learns to extract meaningful features which resemble those found in the human visual cortex. Using a novel parallelization algorithm to distribute the work among multiple machines connected on a network, we show how training such a model can be done in reasonable time. A second problematic aspect of the tiny images dataset is that there are no reliable class labels which makes it hard to use for object recognition experiments. We created two sets of reliable labels. The CIFAR-10 set has 6000 examples of each of 10 classes and the CIFAR-100 set has 600 examples of each of 100 non-overlapping classes. Using these labels, we show that object recognition is signi cantly"}165{"work_id": 4285585446, "title": "Metaverse beyond the hype: Multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy", "abstract": "The metaverse has the potential to extend the physical world using augmented and virtual reality technologies allowing users to seamlessly interact within real and simulated environments using avatars and holograms. Virtual environments and immersive games (such as, Second Life, Fortnite, Roblox and VRChat) have been described as antecedents of the metaverse and offer some insight to the potential socio-economic impact of a fully functional persistent cross platform metaverse. Separating the hype and \u201cmeta\u2026\u201d rebranding from current reality is difficult, as \u201cbig tech\u201d paints a picture of the transformative nature of the metaverse and how it will positively impact people in their work, leisure, and social interaction. The potential impact on the way we conduct business, interact with brands and others, and develop shared experiences is likely to be transformational as the distinct lines between physical and digital are likely to be somewhat blurred from current perceptions. However, although the technology and infrastructure does not yet exist to allow the development of new immersive virtual worlds at scale - one that our avatars could transcend across platforms, researchers are increasingly examining the transformative impact of the metaverse. Impacted sectors include marketing, education, healthcare as well as societal effects relating to social interaction factors from widespread adoption, and issues relating to trust, privacy, bias, disinformation, application of law as well as psychological aspects linked to addiction and impact on vulnerable people. This study examines these topics in detail by combining the informed narrative and multi-perspective approach from experts with varied disciplinary backgrounds on many aspects of the metaverse and its transformational impact. The paper concludes by proposing a future research agenda that is valuable for researchers, professionals and policy makers alike."}166{"work_id": 3187418919, "title": "A Survey on Vision Transformer", "abstract": "Transformer, first applied to the field of natural language processing, is a type of deep neural network mainly based on the self-attention mechanism. Thanks to its strong representation capabilities, researchers are looking at ways to apply transformer to computer vision tasks. In a variety of visual benchmarks, transformer-based models perform similar to or better than other types of networks such as convolutional and recurrent neural networks. Given its high performance and less need for vision-specific inductive bias, transformer is receiving more and more attention from the computer vision community. In this paper, we review these vision transformer models by categorizing them in different tasks and analyzing their advantages and disadvantages. The main categories we explore include the backbone network, high/mid-level vision, low-level vision, and video processing. We also include efficient transformer methods for pushing transformer into real device-based applications. Furthermore, we also take a brief look at the self-attention mechanism in computer vision, as it is the base component in transformer. Toward the end of this paper, we discuss the challenges and provide several further research directions for vision transformers."}167{"work_id": 4408943584, "title": "Digital learning in the 21st century: trends, challenges, and innovations in technology integration", "abstract": "The integration of digital technologies into education represents a significant evolution in the pedagogical landscape, with the potential to enhance accessibility, engagement, and personalization in learning. This review synthesizes current trends, challenges, and innovations within digital learning, emphasizing the impact of artificial intelligence (AI), virtual reality (VR), and online platforms on student achievement. It highlights the importance of addressing technical, pedagogical, and socioeconomic challenges to ensure equitable access to technology. Successful initiatives like the Open University illustrate digital learning's potential to improve educational outcomes. The review also anticipates future directions, including the expanding role of AI, VR, mobile learning, and blockchain in education. It concludes with strategic recommendations for educators and policymakers to adopt best practices, prioritize infrastructure development, and focus on continuous professional development to leverage the benefits of digital learning. As education enters an era of digital transformation, a collaborative approach among stakeholders will be essential in creating an inclusive and effective learning environment for the future."}168{"work_id": 3128261063, "title": "seaborn: statistical data visualization", "abstract": "seaborn is a library for making statistical graphics in Python.It provides a high-level interface to matplotlib and integrates closely with pandas data structures.Functions in the seaborn library expose a declarative, dataset-oriented API that makes it easy to translate questions about data into graphics that can answer them.When given a dataset and a specification of the plot to make, seaborn automatically maps the data values to visual attributes such as color, size, or style, internally computes statistical transformations, and decorates the plot with informative axis labels and a legend.Many seaborn functions can generate figures with multiple panels that elicit comparisons between conditional subsets of data or across different pairings of variables in a dataset.seaborn is designed to be useful throughout the lifecycle of a scientific project.By producing complete graphics from a single function call with minimal arguments, seaborn facilitates rapid prototyping and exploratory data analysis.And by offering extensive options for customization, along with exposing the underlying matplotlib objects, it can be used to create polished, publication-quality figures."}169{"work_id": 2736941579, "title": "Learning Transferable Architectures for Scalable Image Recognition", "abstract": "Developing neural network image classification models often requires significant architecture engineering. In this paper, we study a method to learn the model architectures directly on the dataset of interest. As this approach is expensive when the dataset is large, we propose to search for an architectural building block on a small dataset and then transfer the block to a larger dataset. The key contribution of this work is the design of a new search space (which we call the \"NASNet search space\") which enables transferability. In our experiments, we search for the best convolutional layer (or \"cell\") on the CIFAR-10 dataset and then apply this cell to the ImageNet dataset by stacking together more copies of this cell, each with their own parameters to design a convolutional architecture, which we name a \"NASNet architecture\". We also introduce a new regularization technique called ScheduledDropPath that significantly improves generalization in the NASNet models. On CIFAR-10 itself, a NASNet found by our method achieves 2.4% error rate, which is state-of-the-art. Although the cell is not searched for directly on ImageNet, a NASNet constructed from the best cell achieves, among the published works, state-of-the-art accuracy of 82.7% top-1 and 96.2% top-5 on ImageNet. Our model is 1.2% better in top-1 accuracy than the best human-invented architectures while having 9 billion fewer FLOPS - a reduction of 28% in computational demand from the previous state-of-the-art model. When evaluated at different levels of computational cost, accuracies of NASNets exceed those of the state-of-the-art human-designed models. For instance, a small version of NASNet also achieves 74% top-1 accuracy, which is 3.1% better than equivalently-sized, state-of-the-art models for mobile platforms. Finally, the image features learned from image classification are generically useful and can be transferred to other computer vision problems. On the task of object detection, the learned features by NASNet used with the Faster-RCNN framework surpass state-of-the-art by 4.0% achieving 43.1% mAP on the COCO dataset."}170{"work_id": 7124433690, "title": "RaDe-GS: Rasterizing Depth in Gaussian Splatting", "abstract": "Gaussian Splatting (GS) has proven to be highly effective in novel view synthesis, achieving high-quality and real-time rendering. However, its potential for reconstructing detailed 3D shapes has not been fully explored. Existing methods often suffer from limited shape accuracy due to the discrete and unstructured nature of Gaussian primitives, which complicates the shape extraction. While recent techniques like 2D GS have attempted to improve shape reconstruction, they often reformulate the Gaussian primitives in ways that reduce both rendering quality and computational efficiency. To address these problems, our work introduces a rasterized approach to render the depth maps and surface normal maps of general 3D Gaussian primitives. Our method not only significantly enhances shape reconstruction accuracy but also maintains the computational efficiency intrinsic to Gaussian Splatting. It achieves a Chamfer distance error comparable to Neuralangelo Li et al. [ 2023 ] on the DTU dataset and maintains similar computational efficiency as the original 3D GS methods. Our method is a significant advancement in Gaussian Splatting and can be directly integrated into existing Gaussian Splatting-based methods."}171{"work_id": 3169483174, "title": "mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer", "abstract": "Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, Colin Raffel. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021."}172{"work_id": 3171007011, "title": "Exploring Simple Siamese Representation Learning", "abstract": "Siamese networks have become a common structure in various recent models for unsupervised visual representation learning. These models maximize the similarity between two augmentations of one image, subject to certain conditions for avoiding collapsing solutions. In this paper, we report surprising empirical results that simple Siamese networks can learn meaningful representations even using none of the following: (i) negative sample pairs, (ii) large batches, (iii) momentum encoders. Our experiments show that collapsing solutions do exist for the loss and structure, but a stop-gradient operation plays an essential role in preventing collapsing. We provide a hypothesis on the implication of stop-gradient, and further show proof-of-concept experiments verifying it. Our \"SimSiam\" method achieves competitive results on ImageNet and downstream tasks. We hope this simple baseline will motivate people to rethink the roles of Siamese architectures for unsupervised representation learning. Code is made available.1"}173{"work_id": 7151814497, "title": "Designing tools for 3D content authoring based on 3D sketching", "abstract": "Conception d'outils de cr\u00e9ation de contenu 3D bas\u00e9s sur le dessin 3D L'accessibilit\u00e9 croissante du rendu 3D en temps r\u00e9el a fait de la cr\u00e9ation de contenu 3D un moyen majeur d'expression et de communication. Mais la cr\u00e9ation de contenu 3D n\u00e9cessite d'interagir avec des repr\u00e9sentations num\u00e9riques de la forme et de l'apparence qui sont compatibles avec les algorithmes de rendu et d'animation. Les maillages triangulaires, les mod\u00e8les de mat\u00e9riaux param\u00e9triques et les courbes d'animation sont bien adapt\u00e9s aux op\u00e9rations de rendu mais obligent les artistes \u00e0 exprimer leurs id\u00e9es en terme de commandes bas niveau qui doivent \u00eatre apprises et m\u00e9moris\u00e9es.Dans cette th\u00e8se, nous explorons l'utilisation de coups de crayon 3D ou courbes 3D comme moyen pour les artistes d'exprimer leurs id\u00e9es. Inspir\u00e9s par la fa\u00e7on dont les artistes travaillent avec un pinceau et une toile, nous consid\u00e9rons ce geste de coup de crayon de l'artiste comme la principale commande d'entr\u00e9e du syst\u00e8me de cr\u00e9ation. Les courbes 3D sont des primitives flexibles qui peuvent \u00eatre cr\u00e9\u00e9es dans des interfaces utilisateur 2D ou dans des interfaces de r\u00e9alit\u00e9 virtuelle (RV), et elles peuvent encoder une forme 3D ou l'apparence finale d'une peinture 3D. La conception d'outils qui consid\u00e8rent les courbes 3D comme une repr\u00e9sentation de la forme ou de l'apparence ouvre un espace vaste et passionnant \u00e0 explorer.Les designers peuvent utiliser les courbes 3D comme une repr\u00e9sentation partielle de la forme 3D. Nous \u00e9tudions comment convertir un croquis 3D clairsem\u00e9 en un mod\u00e8le de surface 3D.\u00c9tant donn\u00e9 que les courbes caract\u00e9ristiques de l'objet sont un \u00e9l\u00e9ment important de la forme design\u00e9e et qu'elles sont repr\u00e9sent\u00e9es avec soin dans l'esquisse, nous reconstruisons une surface lisse par morceaux qui pr\u00e9serve ces courbes caract\u00e9ristiques. En obtenant une surface \u00e0 partir des courbes 3D, notre algorithme permet le rendu 3D de la forme d\u00e9crite par l'esquisse.Pour mieux comprendre comment les courbes 3D peuvent repr\u00e9senter non seulement la forme mais aussi l'apparence des objets, nous \u00e9tudions la pratique de la peinture en RV au sein d'une communaut\u00e9 d'artistes qui travaillent avec un logiciel commercial de peinture en RV. Sur la base de cette \u00e9tude, nous proposons une conception et une impl\u00e9mentation pour les \"calques 3D\", une nouvelle primitive d'interaction pour la peinture RV qui consid\u00e8re les courbes 3D comme repr\u00e9sentation \u00e0 la fois de la forme et de l'apparence 3D, tout en d\u00e9couplant l'\u00e9dition de ces deux \u00e9l\u00e9ments. Inspir\u00e9s par l'utilisation de la composition de calques en peinture num\u00e9rique 2D, nous proposons un processus non destructif pour modifier l'apparence d'une peinture RV.L'animation dessin\u00e9e \u00e0 la main est un moyen expressif de cr\u00e9er une animation avec des coups de crayons. Dans les animations de type \"video doodles\", les artistes cr\u00e9ent un dessin anim\u00e9 qui semble bouger dans le m\u00eame espace 3D qu'une vid\u00e9o film\u00e9e. Prendre en compte les effets de perspective et les occlusions tout en dessinant des courbes 2D n'est pas une t\u00e2che facile, c'est pourquoi nous utilisons des techniques de vision par ordinateur pour placer les courbes dans l'espace 3D et les rendre en respectant le contexte de la vid\u00e9o. Nous concevons une interface utilisateur en 2D qui ressemble aux outils traditionnels d'animation en 2D, afin de permettre aux utilisateurs qui ne sont pas familiers avec les outils 3D de cr\u00e9er de telles animations.Globalement, nous montrons que les courbes 3D sont une repr\u00e9sentation puissante pour la cr\u00e9ation de contenu 3D en proposant trois syst\u00e8mes qui exploitent les courbes 3D ou les dessins 3D en tant que primitives d'interaction pour des applications cr\u00e9atives allant de la cr\u00e9ation de forme \u00e0 celle d'apparence et d'animation. Nous abordons la conception de ces syst\u00e8mes sous deux angles compl\u00e9mentaires ; nous d\u00e9veloppons de nouveaux algorithmes pour interpr\u00e9ter les dessins et les commandes de bas niveau de l'utilisateur, et nous concevons des interactions qui permettent aux utilisateurs d'exprimer leurs intentions haut niveau."}174{"work_id": 3100507239, "title": "Illumination-Adaptive Person Re-identification", "abstract": "UTokyo Repository\u306f\u672c\u5b66\u3067\u751f\u7523\u3055\u308c\u305f\u3055\u307e\u3056\u307e\u306a\u5b66\u8853\u6210\u679c\u3092\u96fb\u5b50\u7684\u5f62\u614b\u3067\u96c6\u4e2d\u7684\u306b\u84c4\u7a4d\u30fb\u4fdd\u5b58\u3057\u3001\u4e16\u754c\u306b\u767a\u4fe1\u3059\u308b\u3053\u3068\u3092\u76ee\u7684\u3068\u3057\u305f\u30a4\u30f3\u30bf\u30fc\u30cd\u30c3\u30c8\u4e0a\u306e\u767a\u4fe1\u62e0\u70b9\u3067\u3059\u3002 The UTokyo Repository is the system to store and provide digital resources created by members of the University of Tokyo. Its main purpose is to develop digital collections, make them available online, and preserve them for long-term access."}175{"work_id": 3170841864, "title": "Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers", "abstract": "Most recent semantic segmentation methods adopt a fully-convolutional network (FCN) with an encoder-decoder architecture. The encoder progressively reduces the spatial resolution and learns more abstract/semantic visual concepts with larger receptive fields. Since context modeling is critical for segmentation, the latest efforts have been focused on increasing the receptive field, through either dilated/atrous convolutions or inserting attention modules. However, the encoder-decoder based FCN architecture remains unchanged. In this paper, we aim to provide an alternative perspective by treating semantic segmentation as a sequence-to-sequence prediction task. Specifically, we deploy a pure transformer (i.e., without convolution and resolution reduction) to encode an image as a sequence of patches. With the global context modeled in every layer of the transformer, this encoder can be combined with a simple decoder to provide a powerful segmentation model, termed SEgmentation TRansformer (SETR). Extensive experiments show that SETR achieves new state of the art on ADE20K (50.28% mIoU), Pascal Context (55.83% mIoU) and competitive results on Cityscapes. Particularly, we achieve the first position in the highly competitive ADE20K test server leaderboard on the day of submission."}176{"work_id": 3209984917, "title": "WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing", "abstract": "Self-supervised learning (SSL) achieves great success in speech recognition, while limited exploration has been attempted for other speech processing tasks. As speech signal contains multi-faceted information including speaker identity, paralinguistics, spoken content, etc., learning universal representations for all speech tasks is challenging. To tackle the problem, we propose a new pre-trained model, WavLM, to solve full-stack downstream speech tasks. WavLM jointly learns masked speech prediction and denoising in pre-training. By this means, WavLM does not only keep the speech content modeling capability by the masked speech prediction, but also improves the potential to non-ASR tasks by the speech denoising. In addition, WavLM employs gated relative position bias for the Transformer structure to better capture the sequence ordering of input speech. We also scale up the training dataset from 60 k hours to 94 k hours. WavLM Large achieves state-of-the-art performance on the SUPERB benchmark, and brings significant improvements for various speech processing tasks on their representative benchmarks."}177{"work_id": 4402754006, "title": "DETRs Beat YOLOs on Real-time Object Detection", "abstract": "The YOLO series has become the most popular frame-work for real-time object detection due to its reasonable trade-off between speed and accuracy. However, we observe that the speed and accuracy of YOLOs are negatively affected by the NMS. Recently, end-to-end Transformer-based detectors (DETRs) have provided an alternative to eliminating NMS. Nevertheless, the high computational cost limits their practicality and hinders them from fully exploiting the advantage of excluding NMS. In this paper, we propose the Real-Time DEtection TRansformer (RT-DETR), the first real-time end-to-end object detector to our best knowledge that addresses the above dilemma. We build RT-DETR in two steps, drawing on the advanced DETR: first we focus on maintaining accuracy while improving speed, followed by maintaining speed while improving accuracy. Specifically, we design an efficient hybrid encoder to expeditiously process multi-scale features by decoupling intra-scale interaction and cross-scale fusion to improve speed. Then, we propose the uncertainty-minimal query selection to provide high-quality initial queries to the decoder, thereby improving accuracy. In addition, RT-DETR supports flexible speed tuning by adjusting the number of decoder layers to adapt to various scenarios without retraining. Our RT-DETR-R50 /R101 achieves 53.1% 154.3% AP on COCO and 108 /74 FPS on T4 GPU, outperforming previously advanced YOLOs in both speed and accuracy. Furthermore, RT-DETR-R50 outperforms DINO-R50 by 2.2% AP in accuracy and about 21 times in FPS. After pre-training with Objects365, RT-DETR-R50 / R101 achieves 55.3% 156.2% AP. The project page: https://zhao-yian.github.io/RTDEtr."}178{"work_id": 2963091558, "title": "Non-local Neural Networks", "abstract": "Both convolutional and recurrent operations are building blocks that process one local neighborhood at a time. In this paper, we present non-local operations as a generic family of building blocks for capturing long-range dependencies. Inspired by the classical non-local means method in computer vision, our non-local operation computes the response at a position as a weighted sum of the features at all positions. This building block can be plugged into many computer vision architectures. On the task of video classification, even without any bells and whistles, our non-local models can compete or outperform current competition winners on both Kinetics and Charades datasets. In static image recognition, our non-local models improve object detection/segmentation and pose estimation on the COCO suite of tasks. Code is available at https://github.com/facebookresearch/video-nonlocal-net ."}179{"work_id": 3119997354, "title": "Transformers in Vision: A Survey", "abstract": "Astounding results from Transformer models on natural language tasks have intrigued the vision community to study their application to computer vision problems. Among their salient benefits, Transformers enable modeling long dependencies between input sequence elements and support parallel processing of sequence as compared to recurrent networks, e.g., Long short-term memory. Different from convolutional networks, Transformers require minimal inductive biases for their design and are naturally suited as set-functions. Furthermore, the straightforward design of Transformers allows processing multiple modalities (e.g., images, videos, text, and speech) using similar processing blocks and demonstrates excellent scalability to very large capacity networks and huge datasets. These strengths have led to exciting progress on a number of vision tasks using Transformer networks. This survey aims to provide a comprehensive overview of the Transformer models in the computer vision discipline. We start with an introduction to fundamental concepts behind the success of Transformers, i.e., self-attention, large-scale pre-training, and bidirectional feature encoding. We then cover extensive applications of transformers in vision including popular recognition tasks (e.g., image classification, object detection, action recognition, and segmentation), generative modeling, multi-modal tasks (e.g., visual-question answering, visual reasoning, and visual grounding), video processing (e.g., activity recognition, video forecasting), low-level vision (e.g., image super-resolution, image enhancement, and colorization), and three-dimensional analysis (e.g., point cloud classification and segmentation). We compare the respective advantages and limitations of popular techniques both in terms of architectural design and their experimental value. Finally, we provide an analysis on open research directions and possible future works. We hope this effort will ignite further interest in the community to solve current challenges toward the application of transformer models in computer vision."}180{"work_id": 3043971245, "title": "ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual\u2013Inertial, and Multimap SLAM", "abstract": "This article presents ORB-SLAM3, the first system able to perform visual, visual-inertial and multimap SLAM with monocular, stereo and RGB-D cameras, using pin-hole and fisheye lens models. The first main novelty is a tightly integrated visual-inertial SLAM system that fully relies on maximuma posteriori(MAP) estimation, even during IMU initialization, resulting in real-time robust operation in small and large, indoor and outdoor environments, being two to ten times more accurate than previous approaches. The second main novelty is a multiple map system relying on a new place recognition method with improved recall that lets ORB-SLAM3 survive to long periods of poor visual information: when it gets lost, it starts a new map that will be seamlessly merged with previous maps when revisiting them. Compared with visual odometry systems that only use information from the last few seconds, ORB-SLAM3 is the first system able to reuse in all the algorithm stages all previous information from high parallax co-visible keyframes, even if they are widely separated in time or come from previous mapping sessions, boosting accuracy. Our experiments show that, in all sensor configurations, ORB-SLAM3 is as robust as the best systems available in the literature and significantly more accurate. Notably, our stereo-inertial SLAM achieves an average accuracy of 3.5 cm in the EuRoC drone and 9 mm under quick hand-held motions in the room of TUM-VI dataset, representative of AR/VR scenarios. For the benefit of the community we make public the source code."}181{"work_id": 7123353670, "title": "Trajectory Planning Method for Fracture Reduction of Parallel Robots Based on DMP and APF", "abstract": "The operation of tibial fracture reduction is complex and may involve risks of iatrogenic injuries due to fatigue or insufficient experience of the surgeon. Therefore, this paper proposes a tibial fracture reduction trajectory planning method based on dynamic movement primitives (DMP) and artificial potential fields (APF) to avoid the collision between the distal and proximal fracture fragments during the reduction process and to achieve safe tibial fracture reduction. First, DMP is used to learn the motion characteristics from the orientation demonstration trajectory, generating a new orientation trajectory for the distal fracture fragment. Second, DMP with an obstacle avoidance mechanism based on APF is used to learn the motion characteristics from the position demonstration trajectory, generating a position trajectory for the distal fracture fragment with obstacle avoidance capability. Then, the position trajectory and the orientation trajectory are combined synchronously to form the reduction trajectory. Finally the aforementioned trajectory is optimized to minimize both the the sum of squared velocities and the the sum of squared accelerations of all branch chains under specific constraints, enabling the robot to perform a smooth fracture reduction. Simulations and experiments are conducted, and the results demonstrate that the proposed method can effectively perform trajectory planning for tibial fracture reduction. The proposed method can provide technical references for the trajectory planning and obstacle avoidance problems of orthopedic robots in unstructured environments."}182{"work_id": 7130402520, "title": "How NeRFs and 3-D Gaussian Splatting Are Reshaping SLAM: A Survey", "abstract": "Over the past two decades, research in the field of Simultaneous Localization and Mapping (SLAM) has undergone a significant evolution, highlighting its critical role in enabling autonomous exploration of unknown environments. This evo-lution ranges from hand-crafted methods, through the era of deep learning, to more recent developments focused on Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting (3DGS) rep-resentations. Recognizing the growing body of research and the absence of a comprehensive survey on the topic, this paper aims to provide the first comprehensive overview of SLAM progress through the lens of the latest advancements in radiance fields. It sheds light on the background, evolutionary path, inherent strengths and limitations, and serves as a fundamental reference to highlight the dynamic progress and specific challenges."}183{"work_id": 4403637324, "title": "Promises and challenges of generative artificial intelligence for human learning", "abstract": null}184{"work_id": 4386506836, "title": "5.7 The WORDNET lexical database of English", "abstract": null}185{"work_id": 7126053009, "title": "Information-theoretic graph fusion with vision-language-action model for policy reasoning and dual robotic control", "abstract": null}186{"work_id": 2910064571, "title": "A Practical Guide to Usability Testing", "abstract": null}187{"work_id": 4252097634, "title": "Statistical Shape Analysis", "abstract": null}188{"work_id": 232012359, "title": "IEEE Computer Graphics and Applications", "abstract": null}189{"work_id": 3044864684, "title": "Leveraging orthographic information to improve machine translation of under-resourced languages", "abstract": null}190{"work_id": 4231645407, "title": "The Fastest Texture Mapping in Town:", "abstract": null}191{"work_id": 3017855299, "title": "Automated detection of COVID-19 cases using deep neural networks with X-ray images", "abstract": null}192{"work_id": 2518959197, "title": "Cybersickness prioritization and modeling", "abstract": null}193{"work_id": 2910683382, "title": "Real-Time Simulation and Rendering of 3D Fluids", "abstract": null}194{"work_id": 3030520226, "title": "End-to-End Object Detection with Transformers", "abstract": null}195{"work_id": 2964309882, "title": "Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation", "abstract": null}196{"work_id": 3152382703, "title": "EVOLUTION OF YOLO ALGORITHM AND YOLOV5: THE STATE-OF-THE-ART OBJECT DETECTION ALGORITHM", "abstract": null}197{"work_id": 4385403811, "title": "Diffusion Policy: Visuomotor Policy Learning via Action Diffusion", "abstract": null}198{"work_id": 2915602850, "title": "Memes, Communities and Continuous Change: Chinese Internet Vernacular Explained", "abstract": null}199{"work_id": 2478400128, "title": "Raster scan computer graphics and incremental computing systems", "abstract": null}200{"work_id": 4321232185, "title": "Swin-Unet: Unet-Like Pure Transformer for Medical Image Segmentation", "abstract": null}201{"work_id": 2219249508, "title": "Perceptual Signal Analysis and Eigenvalue-Weighted Metrics for Musical Audio Quality Assessment", "abstract": null}202{"work_id": 4320013936, "title": "Generative Adversarial Nets", "abstract": null}203{"work_id": 4403770406, "title": "YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information", "abstract": null}204{"work_id": 4293774946, "title": "Employing Adaptive Learning and Intelligent Tutoring Robots for Virtual Classrooms and Smart Campuses: Reforming Education in the Age of Artificial Intelligence", "abstract": null}205{"work_id": 4414820664, "title": "How AI and Human Behaviors Shape Psychosocial Effects of Extended Chatbot Use: A Longitudinal Randomized Controlled Study", "abstract": null}206{"work_id": 4404612908, "title": "Grounding DINO: Marrying DINO with Grounded Pre-training for Open-Set Object Detection", "abstract": null}207{"work_id": 3013601031, "title": "Covid-19: automatic detection from X-ray images utilizing transfer learning with convolutional neural networks", "abstract": null}208