Paper Digest: NeurIPS 2022 Highlights
The Conference on Neural Information Processing Systems (NeurIPS) is one of the top machine learning conferences. To help the community quickly catch up on the work presented in this conference, Paper Digest Team processed all accepted papers, and generated one highlight sentence (typically the main topic) for each paper. Readers are encouraged to read these machine generated highlights to quickly get the main idea of each paper.
Note: Neurips-2022 accepts more than 2,900 papers, this page only includes 500 of them selected by our ranking algorithm. Interested users can choose to read All 2,900 NeurIPS-2022 papers in a separate page, which takes quite some time to load.
To search for papers presented at NIPS-2022 on a specific topic, please make use of the search by venue (NIPS-2022) service. To summarize the latest research published at NeurIPS-2022 on a specific topic, you can utilize the review by venue (NIPS-2022) service. If you are interested in browsing papers by author, we have a comprehensive list of ~ 9,900 authors (NIPS-2022). Additionally, you may want to explore our “Best Paper” Digest (NeurIPS), which lists the most influential NeurIPS papers in the past 30 years.
This curated list is created by the Paper Digest Team. Experience the cutting-edge capabilities of Paper Digest, an innovative AI-powered research platform that gets you the personalized and comprehensive updates on the latest research in your field. It also empowers you to read articles, write articles, get answers, conduct literature reviews and generate research reports.
Experience the full potential of our services today!
TABLE 1: Paper Digest: NeurIPS 2022 Highlights
| Paper | Author(s) | |
|---|---|---|
| 1 | Training Language Models to Follow Instructions with Human Feedback Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We fine-tune GPT-3 using data collected from human labelers. The resulting model, called InstructGPT, outperforms GPT-3 on a range of NLP tasks. |
Long Ouyang; Jeffrey Wu; Xu Jiang; Diogo Almeida; Carroll Wainwright; Pamela Mishkin; Chong Zhang; Sandhini Agarwal; Katarina Slama; Alex Ray; John Schulman; Jacob Hilton; Fraser Kelton; Luke Miller; Maddie Simens; Amanda Askell; Peter Welinder; Paul Christiano; Jan Leike; Ryan Lowe; |
| 2 | Chain of Thought Prompting Elicits Reasoning in Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We explore how generating a chain of thought—a series of intermediate reasoning steps—significantly improves the ability of large language models to perform complex reasoning. |
Jason Wei; Xuezhi Wang; Dale Schuurmans; Maarten Bosma; brian ichter; Fei Xia; Ed Chi; Quoc V Le; Denny Zhou; |
| 3 | Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present Imagen, a text-to-image diffusion model with an unprecedented degree of photorealism and a deep level of language understanding. |
Chitwan Saharia; William Chan; Saurabh Saxena; Lala Li; Jay Whang; Emily Denton; Seyed Kamyar Seyed Ghasemipour; Raphael Gontijo Lopes; Burcu Karagol Ayan; Tim Salimans; Jonathan Ho; David Fleet; Mohammad Norouzi; |
| 4 | Large Language Models Are Zero-Shot Reasoners Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a single zero-shot prompt that elicits effective chain of thought reasoning across diverse benchmarks that require multi-step thinking. |
Takeshi Kojima; Shixiang (Shane) Gu; Machel Reid; Yutaka Matsuo; Yusuke Iwasawa; |
| 5 | Flamingo: A Visual Language Model for Few-Shot Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Building models that can be rapidly adapted to novel tasks using only a handful of annotated examples is an open challenge for multimodal machine learning research. We introduce Flamingo, a family of Visual Language Models (VLM) with this ability. |
Jean-Baptiste Alayrac; Jeff Donahue; Pauline Luc; Antoine Miech; Iain Barr; Yana Hasson; Karel Lenc; Arthur Mensch; Katherine Millican; Malcolm Reynolds; Roman Ring; Eliza Rutherford; Serkan Cabi; Tengda Han; Zhitao Gong; Sina Samangooei; Marianne Monteiro; Jacob L Menick; Sebastian Borgeaud; Andy Brock; Aida Nematzadeh; Sahand Sharifzadeh; Mikołaj Bińkowski; Ricardo Barreira; Oriol Vinyals; Andrew Zisserman; Karen Simonyan; |
| 6 | LAION-5B: An Open Large-scale Dataset for Training Next Generation Image-text Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present LAION-5B, an open, publically available dataset of 5.8B image-text pairs and validate it by reproducing results of training state-of-the-art CLIP models of different scale. |
Christoph Schuhmann; Romain Beaumont; Richard Vencu; Cade Gordon; Ross Wightman; Mehdi Cherti; Theo Coombes; Aarush Katta; Clayton Mullis; Mitchell Wortsman; Patrick Schramowski; Srivatsa Kundurthy; Katherine Crowson; Ludwig Schmidt; Robert Kaczmarczyk; Jenia Jitsev; |
| 7 | FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present a fast and memory-efficient exact attention algorithm by accounting for GPU memory reads/writes, yielding faster end-to-end training time and higher quality models with longer sequences. |
Tri Dao; Dan Fu; Stefano Ermon; Atri Rudra; Christopher Ré; |
| 8 | Elucidating The Design Space of Diffusion-Based Generative Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We argue that the theory and practice of diffusion-based generative models are currently unnecessarily convoluted and seek to remedy the situation by presenting a design space that clearly separates the concrete design choices. |
Tero Karras; Miika Aittala; Timo Aila; Samuli Laine; |
| 9 | Video Diffusion Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To generate long and higher resolution videos we introduce a new conditional sampling technique for spatial and temporal video extension that performs better than previously proposed methods. |
Jonathan Ho; Tim Salimans; Alexey Gritsenko; William Chan; Mohammad Norouzi; David Fleet; |
| 10 | DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose an exact formulation of the solution of diffusion ODEs. |
Cheng Lu; Yuhao Zhou; Fan Bao; Jianfei Chen; Chongxuan LI; Jun Zhu; |
| 11 | Locating and Editing Factual Associations in GPT Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We locate and edit the mechanisms underlying factual association within the activations and weights of large pretrained GPT models. |
Kevin Meng; David Bau; Alex Andonian; Yonatan Belinkov; |
| 12 | The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We demonstrate PPO’s effectiveness in popular multi-agent benchmarks and analyze its properties and implementation details through empirical studies. |
Chao Yu; Akash Velu; Eugene Vinitsky; Jiaxuan Gao; Yu Wang; Alexandre Bayen; YI WU; |
| 13 | VideoMAE: Masked Autoencoders Are Data-Efficient Learners for Self-Supervised Video Pre-Training Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we show that video masked autoencoders (VideoMAE) are data-efficient learners for self-supervised video pre-training (SSVP). |
Zhan Tong; Yibing Song; Jue Wang; Limin Wang; |
| 14 | Learn to Explain: Multimodal Reasoning Via Thought Chains for Science Question Answering Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To this end, we present Science Question Answering (SQA), a new benchmark that consists of ~21k multimodal multiple choice questions with a diverse set of science topics and annotations of their answers with corresponding lectures and explanations. |
Pan Lu; Swaroop Mishra; Tanglin Xia; Liang Qiu; Kai-Wei Chang; Song-Chun Zhu; Oyvind Tafjord; Peter Clark; Ashwin Kalyan; |
| 15 | Few-Shot Parameter-Efficient Fine-Tuning Is Better and Cheaper Than In-Context Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a new recipe T-Few for parameter-efficient few-shot learning that outperforms GPT-3 in-context learning. |
Haokun Liu; Derek Tam; Mohammed Muqeeth; Jay Mohta; Tenghao Huang; Mohit Bansal; Colin Raffel; |
| 16 | Diffusion-LM Improves Controllable Text Generation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: While recent works have demonstrated successes on controlling simple sentence attributes (e.g., sentiment), there has been little progress on complex, fine-grained controls (e.g., syntactic structure). To address this challenge, we develop a new non-autoregressive language model based on continuous diffusions that we call Diffusion-LM. |
Xiang Li; John Thickstun; Ishaan Gulrajani; Percy Liang; Tatsunori Hashimoto; |
| 17 | Denoising Diffusion Restoration Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This work addresses these issues by introducing Denoising Diffusion Restoration Models (DDRM), an efficient, unsupervised posterior sampling method. |
Bahjat Kawar; Michael Elad; Stefano Ermon; Jiaming Song; |
| 18 | Solving Quantitative Reasoning Problems with Language Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We train a large Transformer language model on mathematical data and achieve strong performance on quantitative reasoning tasks, including state of the art performance on the MATH dataset. |
Aitor Lewkowycz; Anders Andreassen; Vinay Ramasesh; Henryk Michalewski; David Dohan; Cem Anil; Ambrose Slone; Imanol Schlag; Theo Gutman-Solo; Yuhuai Wu; Ethan Dyer; Guy Gur-Ari; Behnam Neyshabur; Vedant Misra; |
| 19 | Why Do Tree-based Models Still Outperform Deep Learning on Typical Tabular Data? Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Results show that tree-based models remain state-of-the-art on medium-sized data (10K samples) even without accounting for their superior speed. To understand this gap, we conduct an empirical investigation into the differing inductive biases of tree-based models and neural networks. |
Leo Grinsztajn; Edouard Oyallon; Gael Varoquaux; |
| 20 | AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Although the pre-trained Vision Transformers (ViTs) achieved great success in computer vision, adapting a ViT to various image and video tasks is challenging because of its heavy computation and storage burdens, where each model needs to be independently and comprehensively fune-tuned to different tasks, limiting its transferability in different domains. To address this challenge, we propose an effective adaptation approach for Transformer, namely AdaptFormer, which can adapt the pre-trained ViTs into many different image and video tasks efficiently. |
Shoufa Chen; Chongjian GE; Zhan Tong; Jiangliu Wang; Yibing Song; Jue Wang; Ping Luo; |
| 21 | PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: PointNeXt boosts the performance of PointNet++ to the state-of-the-art level with improved training and scaling strategies. |
Guocheng Qian; Yuchen Li; Houwen Peng; Jinjie Mai; Hasan Hammoud; Mohamed Elhoseiny; Bernard Ghanem; |
| 22 | SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present SegNeXt, a simple convolutional network architecture for semantic segmentation. |
Meng-Hao Guo; Cheng-Ze Lu; Qibin Hou; Zhengning Liu; Ming-Ming Cheng; Shi-min Hu; |
| 23 | VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present a unified Vision-Language pretrained Model (VLMo) that jointly learns a dual encoder and a fusion encoder with a modular Transformer network. |
Hangbo Bao; Wenhui Wang; Li Dong; Qiang Liu; Owais Khan Mohammed; Kriti Aggarwal; Songhao Piao; Subhojit Som; Furu Wei; |
| 24 | ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we show the surprisingly good capabilities of plain vision transformers for pose estimation from various aspects, namely simplicity in model structure, scalability in model size, flexibility in training paradigm, and transferability of knowledge between models, through a simple baseline model called ViTPose. |
Yufei Xu; Jing Zhang; Qiming ZHANG; Dacheng Tao; |
| 25 | Masked Autoencoders As Spatiotemporal Learners Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Our study suggests that the general framework of masked autoencoding (BERT, MAE, etc.) can be a unified methodology for representation learning with minimal domain knowledge. |
Christoph Feichtenhofer; haoqi fan; Yanghao Li; Kaiming He; |
| 26 | LION: Latent Point Diffusion Models for 3D Shape Generation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To this end, we introduce the hierarchical Latent Point Diffusion Model (LION) for 3D shape generation. |
xiaohui zeng; Arash Vahdat; Francis Williams; Zan Gojcic; Or Litany; Sanja Fidler; Karsten Kreis; |
| 27 | What Can Transformers Learn In-Context? A Case Study of Simple Function Classes Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: While large language models such as GPT-3 exhibit some ability to perform in-context learning, it is unclear what the relationship is between tasks on which this succeeds and what is present in the training data. To investigate this, we consider the problem of training a model to in-context learn a function class (e.g., linear functions): given data derived from some functions in the class, can we train a model (e.g., a Transformer) to in-context learn most functions from that class? |
Shivam Garg; Dimitris Tsipras; Gregory Valiant; Percy Liang; |
| 28 | MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We demonstrate that depth and normal cues, predicted by general-purpose monocular estimators, significantly improve reconstruction quality and optimization time. |
Zehao Yu; Songyou Peng; Michael Niemeyer; Torsten Sattler; Andreas Geiger; |
| 29 | GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In our work, we aim to train performant 3D generative models that synthesize textured meshes which can be directly consumed by 3D rendering engines, thus immediately usable in downstream applications. |
Jun Gao; Tianchang Shen; Zian Wang; Wenzheng Chen; Kangxue Yin; Daiqing Li; Or Litany; Zan Gojcic; Sanja Fidler; |
| 30 | ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we present an efficient and affordable post-training quantization approach to compress large Transformer-based models, termed as \OURS. |
Zhewei Yao; Reza Yazdani Aminabadi; Minjia Zhang; Xiaoxia Wu; Conglong Li; Yuxiong He; |
| 31 | Beyond Neural Scaling Laws: Beating Power Law Scaling Via Data Pruning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Overall, our work suggests that the discovery of good data-pruning metrics may provide a viable path forward to substantially improved neural scaling laws, thereby reducing the resource costs of modern deep learning. |
Ben Sorscher; Robert Geirhos; Shashank Shekhar; Surya Ganguli; Ari Morcos; |
| 32 | Improving Diffusion Models for Inverse Problems Using Manifold Constraints Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: By studying the generative sampling path, here we show that current solvers throw the sample path off the data manifold, and hence the error accumulates. To address this, we propose an additional correction term inspired by the manifold constraint, which can be used synergistically with the previous solvers to make the iterations close to the manifold. |
Hyungjin Chung; Byeongsu Sim; Dohoon Ryu; Jong Chul Ye; |
| 33 | MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Moreover, most MPNNs only pass two-body messages leading to an intricate relationship between the number of layers and the expressivity of the features. This work introduces MACE, a new equivariant MPNN model that uses higher order messages, and demonstrates that this leads to an improved learning law. |
Ilyes Batatia; David P Kovacs; Gregor Simm; Christoph Ortner; Gabor Csanyi; |
| 34 | WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We develop WebShop – a simulated e-commerce website environment with 1.18 million real-world products and 12, 087 crowd-sourced text instructions. |
Shunyu Yao; Howard Chen; John Yang; Karthik Narasimhan; |
| 35 | STaR: Bootstrapping Reasoning With Reasoning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a technique to iteratively leverage a small number of rationale examples and a large dataset without rationales, to bootstrap the ability to perform successively more complex reasoning. |
Eric Zelikman; Yuhuai Wu; Jesse Mu; Noah Goodman; |
| 36 | BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce a simple LiDAR-camera fusion framework that overcomes the downside of previous fusion approaches. |
Tingting Liang; Hongwei Xie; Kaicheng Yu; Zhongyu Xia; Zhiwei Lin; Yongtao Wang; Tao Tang; Bing Wang; Zhi Tang; |
| 37 | Mind The Gap: Understanding The Modality Gap in Multi-modal Contrastive Representation Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present modality gap, an intriguing geometric phenomenon of the representation space of multi-modal models. |
Victor Weixin Liang; Yuhui Zhang; Yongchan Kwon; Serena Yeung; James Zou; |
| 38 | GPT3.int8(): 8-bit Matrix Multiplication for Transformers at Scale Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We develop methods for Int8 matrix multiplication for transformer multi-layer perceptron (MLP) and attention projection layers, which cut the required memory for inference by half while retaining full precision performance. |
Tim Dettmers; Mike Lewis; Luke Zettlemoyer; |
| 39 | MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: MineDojo is a new framework built on the Minecraft game for developing open-ended, generally capable embodied agents. |
Linxi Fan; Guanzhi Wang; Yunfan Jiang; Ajay Mandlekar; Yuncong Yang; Haoyi Zhu; Andrew Tang; De-An Huang; Yuke Zhu; Anima Anandkumar; |
| 40 | Vision GNN: An Image Is Worth Graph of Nodes Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose to represent the image as a graph structure and introduce a new \emph{Vision GNN} (ViG) architecture to extract graph-level feature for visual tasks. |
Kai Han; Yunhe Wang; Jianyuan Guo; Yehui Tang; Enhua Wu; |
| 41 | Decomposing NeRF for Editing Via Feature Field Distillation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we tackle the problem of semantic scene decomposition of NeRFs to enable query-based local editing of the represented 3D scenes. |
Sosuke Kobayashi; Eiichi Matsumoto; Vincent Sitzmann; |
| 42 | Merging Models with Fisher-Weighted Averaging Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Averaging the parameters of models that have the same architecture and initialization can provide a means of combining their respective capabilities. In this paper, we take the perspective that this ”merging” operation can be seen as choosing parameters that approximately maximize the joint likelihood of the posteriors of the models’ parameters. |
Michael S Matena; Colin Raffel; |
| 43 | CogView2: Faster and Better Text-to-Image Generation Via Hierarchical Transformers Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we put forward a solution based on hierarchical transformers and local parallel autoregressive generation. |
Ming Ding; Wendi Zheng; Wenyi Hong; Jie Tang; |
| 44 | SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interaction Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper presents a novel convolutional neural network for time series forecasting, achieving significant accuracy improvements. |
Minhao LIU; Ailing Zeng; Muxi Chen; Zhijian Xu; Qiuxia LAI; Lingna Ma; Qiang Xu; |
| 45 | Mixture-of-Experts with Expert Choice Routing Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Prior work allocates a fixed number of experts to each token using a top-k function regardless of the relative importance of different tokens. To address this, we propose a heterogeneous mixture-of-experts employing an expert choice method. |
Yanqi Zhou; Tao Lei; Hanxiao Liu; Nan Du; Yanping Huang; Vincent Zhao; Andrew Dai; zhifeng Chen; Quoc V Le; James Laudon; |
| 46 | Machine Learning on Graphs: A Model and Comprehensive Taxonomy Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Here, we aim to bridge the gap between network embedding, graph regularization and graph neural networks. |
Ines Chami; Sami Abu-El-Haija; Bryan Perozzi; Christopher Ré; Kevin Murphy; |
| 47 | GLIPv2: Unifying Localization and Vision-Language Understanding Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present a region-aware vision-language pre-trained model that serves both localization tasks (e.g., object detection, instance segmentation) and understanding (e.g., VQA, image captioning) tasks |
Haotian Zhang; Pengchuan Zhang; Xiaowei Hu; Yen-Chun Chen; Liunian Li; Xiyang Dai; Lijuan Wang; Lu Yuan; Jenq-Neng Hwang; Jianfeng Gao; |
| 48 | SwinTrack: A Simple and Strong Baseline for Transformer Tracking Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we aim to further unleash the power of Transformer by proposing a simple yet efficient fully-attentional tracker, dubbed \textbf{SwinTrack}, within classic Siamese framework. |
Liting Lin; Heng Fan; Zhipeng Zhang; Yong Xu; Haibin Ling; |
| 49 | An Empirical Analysis of Compute-optimal Large Language Model Training Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We investigate the optimal model size and number of tokens for training a transformer language model under a given compute budget. |
Jordan Hoffmann; Sebastian Borgeaud; Arthur Mensch; Elena Buchatskaya; Trevor Cai; Eliza Rutherford; Diego de Las Casas; Lisa Anne Hendricks; Johannes Welbl; Aidan Clark; Thomas Hennigan; Eric Noland; Katherine Millican; George van den Driessche; Bogdan Damoc; Aurelia Guy; Simon Osindero; Karen Simonyan; Erich Elsen; Jack Rae; Oriol Vinyals; Laurent Sifre; |
| 50 | MEMO: Test Time Robustness Via Adaptation and Augmentation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we aim to study and devise methods that make no assumptions about the model training process and are broadly applicable at test time. |
Marvin Zhang; Sergey Levine; Chelsea Finn; |
| 51 | Diagonal State Spaces Are As Effective As Structured State Spaces Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a seq2seq model that uses diagonal state spaces (DSS) for contextualization & delivers state-of-the-art performance on benchmarks requiring long-range reasoning over text, images & audio. |
Ankit Gupta; Albert Gu; Jonathan Berant; |
| 52 | ADBench: Anomaly Detection Benchmark Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Given a long list of anomaly detection algorithms developed in the last few decades, how do they perform with regard to (i) varying levels of supervision, (ii) different types of anomalies, and (iii) noisy and corrupted data? In this work, we answer these key questions by conducting (to our best knowledge) the most comprehensive anomaly detection benchmark with 30 algorithms on 57 benchmark datasets, named ADBench. |
Songqiao Han; Xiyang Hu; Hailiang Huang; Minqi Jiang; Yue Zhao; |
| 53 | Video PreTraining (VPT): Learning to Act By Watching Unlabeled Online Videos Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We extend the internet-scale pretraining paradigm to sequential decision domains through semi-supervised imitation learning wherein agents learn to act by watching online unlabeled videos. |
Bowen Baker; Ilge Akkaya; Peter Zhokov; Joost Huizinga; Jie Tang; Adrien Ecoffet; Brandon Houghton; Raul Sampedro; Jeff Clune; |
| 54 | Flexible Diffusion Modeling of Long Videos Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present a framework for video modeling based on denoising diffusion probabilistic models that produces long-duration video completions in a variety of realistic environments. |
William Harvey; Saeid Naderiparizi; Vaden Masrani; Christian Weilbach; Frank Wood; |
| 55 | On The Parameterization and Initialization of Diagonal State Space Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This work seeks to systematically understand how to parameterize and initialize diagonal state space models. |
Albert Gu; Karan Goel; Ankit Gupta; Christopher Ré; |
| 56 | Masked Autoencoders That Listen Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Audio-MAE learns SoTA embeddings from audio spectrograms. Without external pretraining, it achieves best performance with high masking ratio (80%) and decoders with local attention. Qualitative audible reconstructions demonstrate its effectiveness. |
Po-Yao Huang; Hu Xu; Juncheng Li; Alexei Baevski; Michael Auli; Wojciech Galuba; Florian Metze; Christoph Feichtenhofer; |
| 57 | Self-Supervised Contrastive Pre-Training For Time Series Via Time-Frequency Consistency Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To this end, we posit that time-frequency consistency (TF-C) — embedding a time-based neighborhood of a particular example close to its frequency-based neighborhood and back — is desirable for pre-training. |
Xiang Zhang; Ziyuan Zhao; Theodoros Tsiligkaridis; Marinka Zitnik; |
| 58 | Torsional Diffusion for Molecular Conformer Generation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose torsional diffusion, a novel diffusion framework that operates on the space of torsion angles via a diffusion process on the hypertorus and an extrinsic-to-intrinsic score model. |
Bowen Jing; Gabriele Corso; Jeffrey Chang; Regina Barzilay; Tommi Jaakkola; |
| 59 | Transformer Memory As A Differentiable Search Index Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To this end, we introduce the Differentiable Search Index (DSI), a new paradigm that learns a text-to-text model that maps string queries directly to relevant docids; in other words, a DSI model answers queries directly using only its parameters, dramatically simplifying the whole retrieval process. |
Yi Tay; Vinh Tran; Mostafa Dehghani; Jianmo Ni; Dara Bahri; Harsh Mehta; Zhen Qin; Kai Hui; Zhe Zhao; Jai Gupta; Tal Schuster; William Cohen; Donald Metzler; |
| 60 | Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose test-time prompt tuning (TPT) for CLIP to improve its zero-shot generalization. Our method works on a single test sample without the need for training data or annotations. |
Manli Shu; Chaowei Xiao; Weili Nie; De-An Huang; Zhiding Yu; Tom Goldstein; Anima Anandkumar; |
| 61 | AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To mitigate the limitations, we present AMOS, a large-scale, diverse, clinical dataset for abdominal organ segmentation. |
Yuanfeng Ji; Haotian Bai; Chongjian GE; Jie Yang; Ye Zhu; Ruimao Zhang; Zhen Li; Lingyan Zhanng; Wanling Ma; Xiang Wan; Ping Luo; |
| 62 | Unifying Voxel-based Representation with Transformer for 3D Object Detection Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we present a unified framework for multi-modality 3D object detection, named UVTR. |
Yanwei Li; Yilun Chen; Xiaojuan Qi; Zeming Li; Jian Sun; Jiaya Jia; |
| 63 | HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutions Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present the Recursive Gated Convolution ($\textit{g}^\textit{n}$Conv) that performs high-order spatial interactions with gated convolutions and recursive designs. |
Yongming Rao; Wenliang Zhao; Yansong Tang; Jie Zhou; Ser Nam Lim; Jiwen Lu; |
| 64 | Focal Modulation Networks Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose focal modulation network (FocalNet in short), where self-attention (SA) is completely replaced by a focal modulation module that is more effective and efficient for modeling token interactions. |
Jianwei Yang; Chunyuan Li; Xiyang Dai; Jianfeng Gao; |
| 65 | Pre-Trained Language Models for Interactive Decision-Making Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose an approach for using LMs to scaffold learning and generalization in general sequential decision-making problems. |
Shuang Li; Xavier Puig; Chris Paxton; Yilun Du; Clinton Wang; Linxi Fan; Tao Chen; De-An Huang; Ekin Akyürek; Anima Anandkumar; Jacob Andreas; Igor Mordatch; Antonio Torralba; Yuke Zhu; |
| 66 | Data Distributional Properties Drive Emergent In-Context Learning in Transformers Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This observation raises the question: what aspects of the training regime lead to this emergent behavior? Here, we show that this behavior is driven by the distributions of the training data itself. |
Stephanie Chan; Adam Santoro; Andrew Lampinen; Jane Wang; Aaditya Singh; Pierre Richemond; James McClelland; Felix Hill; |
| 67 | GhostNetV2: Enhance Cheap Operation with Long-Range Attention Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose a hardware-friendly attention mechanism (dubbed DFC attention) and then present a new GhostNetV2 architecture for mobile applications. |
Yehui Tang; Kai Han; Jianyuan Guo; Chang Xu; Chao Xu; Yunhe Wang; |
| 68 | Generating Training Data with Language Models: Towards Zero-Shot Language Understanding Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we present a simple approach that uses both types of PLMs for fully zero-shot learning of NLU tasks without requiring any task-specific data: A unidirectional PLM generates class-conditioned texts guided by prompts, which are used as the training data for fine-tuning a bidirectional PLM. |
Yu Meng; Jiaxin Huang; Yu Zhang; Jiawei Han; |
| 69 | Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-training Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose Point-M2AE, a strong Multi-scale MAE pre-training framework for hierarchical self-supervised learning of 3D point clouds. |
Renrui Zhang; Ziyu Guo; Peng Gao; Rongyao Fang; Bin Zhao; Dong Wang; Yu Qiao; Hongsheng Li; |
| 70 | A Contrastive Framework for Neural Text Generation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present a contrastive solution: (i) SimCTG, a contrastive training objective to calibrate the model’s representation space, and (ii) a decoding method—contrastive search—to encourage diversity while maintaining coherence in the generated text. |
Yixuan Su; Tian Lan; Yan Wang; Dani Yogatama; Lingpeng Kong; Nigel Collier; |
| 71 | LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This is because the gradient computation for the trainable parameters still requires backpropagation through the large pre-trained backbone model. To address this, we propose Ladder Side-Tuning (LST), a new PETL technique that can also reduce training memory requirements by more substantial amounts. |
Yi-Lin Sung; Jaemin Cho; Mohit Bansal; |
| 72 | CodeRL: Mastering Code Generation Through Pretrained Models and Deep Reinforcement Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce CodeRL, a novel framework for program synthesis through large-scale pretrained models and deep reinforcement learning and obtained new SOTA on both the challenging APPS and MBPP benchmarks. |
Hung Le; Yue Wang; Akhilesh Deepak Gotmare; Silvio Savarese; Steven Chu Hong Hoi; |
| 73 | Efficient and Modular Implicit Differentiation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we proposeautomatic implicit differentiation, an efficientand modular approach for implicit differentiation of optimization problems. |
Mathieu Blondel; Quentin Berthet; Marco Cuturi; Roy Frostig; Stephan Hoyer; Felipe Llinares-Lopez; Fabian Pedregosa; Jean-Philippe Vert; |
| 74 | Zero-Shot Video Question Answering Via Frozen Bidirectional Language Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present a framework based on frozen bidirectional masked language models to tackle zero-shot video question answering. |
Antoine Yang; Antoine Miech; Josef Sivic; Ivan Laptev; Cordelia Schmid; |
| 75 | OpenOOD: Benchmarking Generalized Out-of-Distribution Detection Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We build an open-source codebase called OpenOOD to support and compare 30+ methods for OOD detection and beyond. |
Jingkang Yang; Pengyun Wang; Dejian Zou; Zitang Zhou; Kunyuan Ding; WENXUAN PENG; Haoqi Wang; Guangyao Chen; Bo Li; Yiyou Sun; Xuefeng Du; Kaiyang Zhou; Wayne Zhang; Dan Hendrycks; Yixuan Li; Ziwei Liu; |
| 76 | Multi-Game Decision Transformers Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In the subfields of vision and language, this was largely achieved by scaling up transformer-based models and training them on large, diverse datasets. Motivated by this progress, we investigate whether the same strategy can be used to produce generalist reinforcement learning agents. |
Kuang-Huei Lee; Ofir Nachum; Mengjiao (Sherry) Yang; Lisa Lee; Daniel Freeman; Sergio Guadarrama; Ian Fischer; Winnie Xu; Eric Jang; Henryk Michalewski; Igor Mordatch; |
| 77 | Diffusion Models As Plug-and-Play Priors Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We consider the problem of inferring high-dimensional data $x$ in a model that consists of a prior $p(x)$ and an auxiliary constraint $c(x,y)$. |
Alexandros Graikos; Nikolay Malkin; Nebojsa Jojic; Dimitris Samaras; |
| 78 | Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we introduce a new compression framework which covers both weight pruning and quantization in a unified setting, is time- and space-efficient, and considerably improves upon the practical performance of existing post-training methods. |
Elias Frantar; Dan Alistarh; |
| 79 | Knowledge Distillation from A Stronger Teacher Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Unlike existing knowledge distillation methods focus on the baseline settings, where the teacher models and training strategies are not that strong and competing as state-of-the-art approaches, this paper presents a method dubbed DIST to distill better from a stronger teacher. |
Tao Huang; Shan You; Fei Wang; Chen Qian; Chang Xu; |
| 80 | QUARK: Controllable Text Generation with Reinforced Unlearning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce Quantized Reward Konditioning (Quark), an algorithm for optimizing a reward function that quantifies an (un)wanted property, while not straying too far from the original model. |
Ximing Lu; Sean Welleck; Liwei Jiang; Jack Hessel; Lianhui Qin; Peter West; Prithviraj Ammanabrolu; Yejin Choi; |
| 81 | SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we present SatMAE, a pre-training framework for temporal or multi-spectral satellite imagery based on Masked Autoencoder (MAE). |
Samar Khanna; Yezhen Cong; Chenlin Meng; Patrick Liu; Erik Rozi; Yutong He; Marshall Burke; David Lobell; Stefano Ermon; |
| 82 | Where2comm: Communication-Efficient Collaborative Perception Via Spatial Confidence Maps Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: It inevitably results in a fundamental trade-off between perception performance and communication bandwidth. To tackle this bottleneck issue, we propose a spatial confidence map, which reflects the spatial heterogeneity of perceptual information. |
Yue Hu; Shaoheng Fang; Zixing Lei; Yiqi Zhong; Siheng Chen; |
| 83 | CLIPDraw: Exploring Text-to-Drawing Synthesis Through Language-Image Encoders Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: CLIPDraw is an algorithm that synthesizes novel drawings from natural language input. |
Kevin Frans; Olaf Witkowski; Lisa Soros; |
| 84 | Factuality Enhanced Language Models for Open-Ended Text Generation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a factual-nucleus sampling algorithm that dynamically adapts the randomness to improve the factuality of generation while maintaining quality. |
Nayeon Lee; Wei Ping; Peng Xu; Mostofa Patwary; Mohammad Shoeybi; Bryan Catanzaro; |
| 85 | Towards Robust Blind Face Restoration with Codebook Lookup Transformer Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we demonstrate that the uncertainty and ambiguity of the mapping can be largely reduced by casting face restoration as a code prediction task in a small, finite proxy feature space. Under this paradigm, we propose a Transformer-based prediction network, named \textit{CodeFormer}, to exploit global contexts of the input for \textit{code prediction}, enabling the discovery of a natural face that closely approximates the target high-quality image even when the input is severely degraded. |
Shangchen Zhou; Kelvin Chan; Chongyi Li; Chen Change Loy; |
| 86 | Geo-Neus: Geometry-Consistent Neural Implicit Surfaces Learning for Multi-view Reconstruction Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: However, one key challenge remains: existing approaches lack explicit multi-view geometry constraints, hence usually fail to generate geometry consistent surface reconstruction. To address this challenge, we propose geometry-consistent neural implicit surfaces learning for multi-view reconstruction. |
Qiancheng Fu; Qingshan Xu; Yew Soon Ong; Wenbing Tao; |
| 87 | Pure Transformers Are Powerful Graph Learners Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We show that standard Transformers without graph-specific modifications can work well in graph learning both in theory and practice. |
Jinwoo Kim; Dat Nguyen; Seonwoo Min; Sungjun Cho; Moontae Lee; Honglak Lee; Seunghoon Hong; |
| 88 | Deep Bidirectional Language-Knowledge Graph Pretraining Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Here we propose DRAGON (Deep Bidirectional Language-Knowledge Graph Pretraining), a self-supervised approach to pretraining a deeply joint language-knowledge model from raw text and KG at scale. |
Michihiro Yasunaga; Antoine Bosselut; Hongyu Ren; Xikun Zhang; Christopher D Manning; Percy Liang; Jure Leskovec; |
| 89 | Long Range Graph Benchmark Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present the Long Range Graph Benchmark (LRGB) with 5 datasets that can be used for the development of models enabling long range dependencies in graphs, like Graph Transformers. |
Vijay Prakash Dwivedi; Ladislav Rampášek; Mikhail Galkin; Ali Parviz; Guy Wolf; Anh Tuan Luu; Dominique Beaini; |
| 90 | PDEBench: An Extensive Benchmark for Scientific Machine Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We provide a benckmark for Scientific Machine Learning |
Makoto Takamoto; Timothy Praditia; Raphael Leiteritz; Daniel MacKinlay; Francesco Alesiani; Dirk Pflüger; Mathias Niepert; |
| 91 | A Unified Model for Multi-class Anomaly Detection Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present UniAD that accomplishes anomaly detection for multiple classes with a unified framework. |
Zhiyuan You; Lei Cui; Yujun Shen; Kai Yang; Xin Lu; Yu Zheng; Xinyi Le; |
| 92 | EGSDE: Unpaired Image-to-Image Translation Via Energy-Guided Stochastic Differential Equations Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To this end, we propose energy-guided stochastic differential equations (EGSDE) that employs an energy function pretrained on both the source and target domains to guide the inference process of a pretrained SDE for realistic and faithful unpaired I2I. |
Min Zhao; Fan Bao; Chongxuan LI; Jun Zhu; |
| 93 | Inception Transformer Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Recent studies show that transformer has strong capability of building long-range dependencies, yet is incompetent in capturing high frequencies that predominantly convey local information. To tackle this issue, we present a novel and general-purpose $\textit{Inception Transformer}$, or $\textit{iFormer}$ for short, that effectively learns comprehensive features with both high- and low-frequency information in visual data. |
Chenyang Si; Weihao Yu; Pan Zhou; Yichen Zhou; Xinchao Wang; Shuicheng Yan; |
| 94 | Egocentric Video-Language Pretraining Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We pioneer Egocentric Video-Language Pretraining from pretraining dataset, model and development benchmark; the resulted pretrained model exhibits strong performance on five downstream tasks across three egocentric datasets. |
Kevin Qinghong Lin; Jinpeng Wang; Mattia Soldan; Michael Wray; Rui Yan; Eric Z. XU; Denial Gao; Rong-Cheng Tu; Wenzhe Zhao; Weijie Kong; Chengfei Cai; WANG HongFa; Dima Damen; Bernard Ghanem; Wei Liu; Mike Zheng Shou; |
| 95 | Memorization Without Overfitting: Analyzing The Training Dynamics of Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We empirically study exact memorization in causal and masked language modeling, across model sizes and throughout the training process. |
Kushal Tirumala; Aram Markosyan; Luke Zettlemoyer; Armen Aghajanyan; |
| 96 | On-Device Training Under 256KB Memory Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose an algorithm-system co-design framework to make training neural networks possible with only 256KB of memory. |
Ji Lin; Ligeng Zhu; Wei-Ming Chen; Wei-Chen Wang; Chuang Gan; Song Han; |
| 97 | Multimodal Contrastive Learning with LIMoE: The Language-Image Mixture of Experts Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We develop a multimodal, sparsely activated Mixture of Experts model, trained contrastively on Image and Text, proposing new regularisation schemes to stabilize it, and significantly outperform dense baselines. |
Basil Mustafa; Carlos Riquelme; Joan Puigcerver; Rodolphe Jenatton; Neil Houlsby; |
| 98 | Revisiting Heterophily For Graph Neural Networks Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we first revisit the widely used homophily metrics and point out that their consideration of only graph-label consistency is a shortcoming. Then, we study heterophily from the perspective of post-aggregation node similarity and define new homophily metrics, which are potentially advantageous compared to existing ones. |
Sitao Luan; Chenqing Hua; Qincheng Lu; Jiaqi Zhu; Mingde Zhao; Shuyuan Zhang; Xiao-Wen Chang; Doina Precup; |
| 99 | Neural Sheaf Diffusion: A Topological Perspective on Heterophily and Oversmoothing in GNNs Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we use cellular sheaf theory to show that the underlying geometry of the graph is deeply linked with the performance of GNNs in heterophilic settings and their oversmoothing behaviour. |
Cristian Bodnar; Francesco Di Giovanni; Benjamin Chamberlain; Pietro Lió; Michael Bronstein; |
| 100 | Multi-Agent Reinforcement Learning Is A Sequence Modeling Problem Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we introduce a novel architecture named Multi-Agent Transformer (MAT) that effectively casts cooperative multi-agent reinforcement learning (MARL) into SM problems wherein the objective is to map agents’ observation sequences to agents’ optimal action sequences. |
Muning Wen; Jakub Kuba; Runji Lin; Weinan Zhang; Ying Wen; Jun Wang; Yaodong Yang; |
| 101 | FiLM: Frequency Improved Legendre Memory Model for Long-term Time Series Forecasting Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To this end, we design a \textbf{F}requency \textbf{i}improved \textbf{L}egendre \textbf{M}emory model, or {\bf FiLM}: it applies Legendre polynomial projections to approximate historical information, uses Fourier projection to remove noise, and adds a low-rank approximation to speed up computation. |
Tian Zhou; Ziqing MA; xue wang; Qingsong Wen; Liang Sun; Tao Yao; Wotao Yin; Rong Jin; |
| 102 | Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple Yet Strong Baseline Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Current end-to-end autonomous driving methods either run a controller based on a planned trajectory or perform control prediction directly, which have spanned two separately studied lines of research. Seeing their potential mutual benefits to each other, this paper takes the initiative to explore the combination of these two well-developed worlds. |
Penghao Wu; Xiaosong Jia; Li Chen; Junchi Yan; Hongyang Li; Yu Qiao; |
| 103 | Trajectory Balance: Improved Credit Assignment in GFlowNets Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We thus propose a new learning objective for GFlowNets, trajectory balance, as a more efficient alternative to previously used objectives. |
Nikolay Malkin; Moksh Jain; Emmanuel Bengio; Chen Sun; Yoshua Bengio; |
| 104 | Patching Open-vocabulary Models By Interpolating Weights Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We study model patching, where the goal is to improve accuracy on specific tasks (i.e., patching tasks) without degrading accuracy on tasks where performance is already adequate (i.e., supported tasks). |
Gabriel Ilharco; Mitchell Wortsman; Samir Yitzhak Gadre; Shuran Song; Hannaneh Hajishirzi; Simon Kornblith; Ali Farhadi; Ludwig Schmidt; |
| 105 | Federated Learning from Pre-Trained Models: A Contrastive Learning Approach Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: However, excessive computation and communication demands pose challenges to current FL frameworks, especially when training large-scale models. To prevent these issues from hindering the deployment of FL systems, we propose a lightweight framework where clients jointly learn to fuse the representations generated by multiple fixed pre-trained models rather than training a large-scale model from scratch. |
Yue Tan; Guodong Long; Jie Ma; LU LIU; Tianyi Zhou; Jing Jiang; |
| 106 | Exploring Length Generalization in Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We explore the ability of transformer-based language models to learn from shorter problem instances to generalize to longer ones and identify points of failure and success. |
Cem Anil; Yuhuai Wu; Anders Andreassen; Aitor Lewkowycz; Vedant Misra; Vinay Ramasesh; Ambrose Slone; Guy Gur-Ari; Ethan Dyer; Behnam Neyshabur; |
| 107 | Autoformalization with Large Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Large language models can be used to do autoformalization, allowing us to achieve in a new SOTA on miniF2F benchmark. |
Yuhuai Wu; Albert Qiaochu Jiang; Wenda Li; Markus N Rabe; Charles Staats; Mateja Jamnik; Christian Szegedy; |
| 108 | Confident Adaptive Language Modeling Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we introduce Confident Adaptive Language Modeling (CALM), a framework for dynamically allocating different amounts of compute per input and generation timestep. |
Tal Schuster; Adam Fisch; Jai Gupta; Mostafa Dehghani; Dara Bahri; Vinh Tran; Yi Tay; Donald Metzler; |
| 109 | Autoregressive Search Engines: Generating Substrings As Document Identifiers Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Previous work has explored ways to partition the search space into hierarchical structures and retrieve documents by autoregressively generating their unique identifier. In this work we propose an alternative that doesn’t force any structure in the search space: using all ngrams in a passage as its possible identifiers. |
Michele Bevilacqua; Giuseppe Ottaviano; Patrick Lewis; Scott Yih; Sebastian Riedel; Fabio Petroni; |
| 110 | Shape, Light, and Material Decomposition from Images Using Monte Carlo Rendering and Denoising Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present an efficient method to jointly reconstruct geometry (explicit triangle meshes), materials, and lighting, which substantially improves material and light separation compared to previous work. |
Jon Hasselgren; Nikolai Hofmann; Jacob Munkberg; |
| 111 | Test-Time Training with Masked Autoencoders Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We show how applying masked autoencoding to train on each unlabeled test sample before making a prediction improves generalization. |
Yossi Gandelsman; Yu Sun; Xinlei Chen; Alexei Efros; |
| 112 | End-to-end Symbolic Regression with Transformers Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: One can subsequently refine the predicted constants by feeding them to the non-convex optimizer as an informed initialization. We present ablations to show that this end-to-end approach yields better results, sometimes even without the refinement step. |
Pierre-alexandre Kamienny; Stéphane d’Ascoli; Guillaume Lample; Francois Charton; |
| 113 | OmniVL: One Foundation Model for Image-Language and Video-Language Tasks Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper presents OmniVL, a new foundation model to support both image-language and video-language tasks using one universal architecture. |
Junke Wang; Dongdong Chen; Zuxuan Wu; Chong Luo; Luowei Zhou; Yucheng Zhao; Yujia Xie; Ce Liu; Yu-Gang Jiang; Lu Yuan; |
| 114 | On Embeddings for Numerical Features in Tabular Deep Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we argue that embeddings for numerical features are an underexplored degree of freedom in tabular DL, which allows constructing more powerful DL models and competing with gradient boosted decision trees (GBDT) on some GBDT-friendly benchmarks (that is, where GBDT outperforms conventional DL models). |
Yury Gorishniy; Ivan Rubachev; Artem Babenko; |
| 115 | Dataset Distillation Using Neural Feature Regression Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Meta-gradient computation is one of the key challenges in this formulation, as differentiating through the inner loop learning procedure introduces significant computation and memory costs. In this paper, we address these challenges using neural Feature Regression with Pooling (FRePo), achieving the state-of-the-art performance with an order of magnitude less memory requirement and two orders of magnitude faster training than previous methods. |
Yongchao Zhou; Ehsan Nezhadarya; Jimmy Ba; |
| 116 | VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Grids Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper investigates sparse voxel grids as 3D representation for 3D-aware image synthesis to achieve efficient rendering. |
Katja Schwarz; Axel Sauer; Michael Niemeyer; Yiyi Liao; Andreas Geiger; |
| 117 | Ensemble of Averages: Improving Model Selection and Boosting Performance in Domain Generalization Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: A simple hyper-parameter free strategy of using the simple moving average of model parameters during training and ensembling achieves SOTA on domain generalization benchmarks, and can be explained using the Bias-Variance trade-off. |
Devansh Arpit; Huan Wang; Yingbo Zhou; Caiming Xiong; |
| 118 | ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: ELEVATER provides the first public platform and toolkit to evaluate vision foundation models in their large-scale task-level visual transfer in 20 image classification tasks and 35 object detection tasks |
Chunyuan Li; Haotian Liu; Liunian Li; Pengchuan Zhang; Jyoti Aneja; Jianwei Yang; Ping Jin; Houdong Hu; Zicheng Liu; Yong Jae Lee; Jianfeng Gao; |
| 119 | Fast Vision Transformers with HiLo Attention Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Thus, we propose to use the direct speed evaluation on the target platform as the design principle for efficient ViTs. |
Zizheng Pan; Jianfei Cai; Bohan Zhuang; |
| 120 | Earthformer: Exploring Space-Time Transformers for Earth System Forecasting Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose \emph{Earthformer}, a space-time Transformer for Earth system forecasting. |
Zhihan Gao; Xingjian Shi; Hao Wang; Yi Zhu; Yuyang (Bernie) Wang; Mu Li; Dit-Yan Yeung; |
| 121 | Recurrent Video Restoration Transformer with Guided Deformable Attention Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we attempt to integrate the advantages of the two cases by proposing a recurrent video restoration transformer, namely RVRT. |
Jingyun Liang; Yuchen Fan; Xiaoyu Xiang; Rakesh Ranjan; Eddy Ilg; Simon Green; Jiezhang Cao; Kai Zhang; Radu Timofte; Luc V Gool; |
| 122 | A Unified Sequence Interface for Vision Tasks Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work we show that a diverse set of “core” computer vision tasks can also be unified if formulated in terms of a shared pixel-to-sequence interface. |
Ting Chen; Saurabh Saxena; Lala Li; Tsung-Yi Lin; David Fleet; Geoffrey E Hinton; |
| 123 | Decoupling Features in Hierarchical Propagation for Video Object Segmentation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper focuses on developing a more effective method of hierarchical propagation for semi-supervised Video Object Segmentation (VOS). |
Zongxin Yang; Yi Yang; |
| 124 | DetCLIP: Dictionary-Enriched Visual-Concept Paralleled Pre-training for Open-world Detection Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper presents DetCLIP, a paralleled visual-concept pre-training method for open-world detection by resorting to knowledge enrichment from a designed concept dictionary. |
Lewei Yao; Jianhua Han; Youpeng Wen; Xiaodan Liang; Dan Xu; Wei Zhang; Zhenguo Li; Chunjing XU; Hang Xu; |
| 125 | SAVi++: Towards End-to-End Object-Centric Learning from Real-World Videos Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce SAVi++, an object-centric video model which is trained to predict depth signals from a slot-based video representation. SAVi++ is able to learn emergent object segmentation and tracking from videos in the real-world Waymo Open dataset. |
Gamaleldin Elsayed; Aravindh Mahendran; Sjoerd van Steenkiste; Klaus Greff; Michael Mozer; Thomas Kipf; |
| 126 | COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we present Energy-based Constrained Decoding with Langevin Dynamics (Cold), a decoding framework which unifies constrained generation as specifying constraints through an energy function, then performing efficient differentiable reasoning over the constraints through gradient-based sampling. |
Lianhui Qin; Sean Welleck; Daniel Khashabi; Yejin Choi; |
| 127 | S4ND: Modeling Images and Videos As Multidimensional Signals with State Spaces Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose a multidimensional version of S4 for modeling visual data. |
Eric Nguyen; Karan Goel; Albert Gu; Gordon Downs; Preey Shah; Tri Dao; Stephen Baccus; Christopher Ré; |
| 128 | A Fast Post-Training Pruning Framework for Transformers Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To retain high accuracy without retraining, we introduce three novel techniques: (i) a lightweight mask search algorithm that finds which heads and filters to prune based on the Fisher information; (ii) mask rearrangement that complements the search algorithm; and (iii) mask tuning that reconstructs the output activations for each layer. |
Woosuk Kwon; Sehoon Kim; Michael Mahoney; Joseph Hassoun; Kurt Keutzer; Amir Gholami; |
| 129 | Contrastive Learning As Goal-Conditioned Reinforcement Learning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, instead of adding representation learning parts to an existing RL algorithm, we show (contrastive) representation learning methods are already RL algorithms in their own right. |
Benjamin Eysenbach; Tianjun Zhang; Sergey Levine; Russ Salakhutdinov; |
| 130 | Reconstructing Training Data From Trained Neural Networks Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We provide a novel scheme for reconstructing large portions of the actual training samples from a trained neural network. Our scheme is inspired by recent theoretical results of the implicit bias in training neural networks. |
Niv Haim; Gal Vardi; Gilad Yehudai; Michal Irani; Ohad Shamir; |
| 131 | ZSON: Zero-Shot Object-Goal Navigation Using Multimodal Goal Embeddings Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose using a vision-and-language model to create semantic representations of navigation goals to enable zero-shot open-world ObjectNav. |
Arjun Majumdar; Gunjan Aggarwal; Bhavika Devnani; Judy Hoffman; Dhruv Batra; |
| 132 | TANKBind: Trigonometry-Aware Neural NetworKs for Drug-Protein Binding Structure Prediction Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we propose Trigonometry-Aware Neural networKs for binding structure prediction, TANKBind, that builds trigonometry constraint as a vigorous inductive bias into the model and explicitly attends to all possible binding sites for each protein by segmenting the whole protein into functional blocks. |
Wei Lu; Qifeng Wu; Jixian Zhang; Jiahua Rao; Chengtao Li; Shuangjia Zheng; |
| 133 | BackdoorBench: A Comprehensive Benchmark of Backdoor Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We also provide comprehensive evaluations of every pair of 8 attacks against 9 defenses, with 5 poisoning ratios, based on 5 models and 4 datasets, thus 8,000 pairs of evaluations in total. We present abundant analysis from different perspectives about these 8,000 evaluations, studying the effects of different factors in backdoor learning. |
Baoyuan Wu; Hongrui Chen; Mingda Zhang; Zihao Zhu; Shaokui Wei; Danni Yuan; Chao Shen; |
| 134 | GAUDI: A Neural Architect for Immersive 3D Scene Generation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce GAUDI, a generative model capable of capturing the distribution of complex and realistic 3D scenes that can be rendered immersively from a moving camera. |
Miguel Angel Bautista; Pengsheng Guo; Samira Abnar; Walter Talbott; Alexander Toshev; Zhuoyuan Chen; Laurent Dinh; Shuangfei Zhai; Hanlin Goh; Daniel Ulbricht; Afshin Dehghan; Joshua Susskind; |
| 135 | Outlier Suppression: Pushing The Limit of Low-bit Transformer Language Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We discover that $\boldsymbol \gamma$ in LayerNorm (LN) acts as a sinful amplifier for the outliers, and the importance of outliers varies greatly where some outliers provided by a few tokens cover a large area but can be clipped sharply without negative impacts. Motivated by these findings, we propose an outlier suppression framework including two components: Gamma Migration and Token-Wise Clipping. |
Xiuying Wei; Yunchen Zhang; Xiangguo Zhang; Ruihao Gong; Shanghang Zhang; Qi Zhang; Fengwei Yu; Xianglong Liu; |
| 136 | Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphs Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Accordingly, we propose an information-theoretic objective to extract the desired subgraphs that maximally preserve the invariant intra-class information. |
Yongqiang Chen; Yonggang Zhang; Yatao Bian; Han Yang; MA Kaili; Binghui Xie; Tongliang Liu; Bo Han; James Cheng; |
| 137 | A Neural Corpus Indexer for Document Retrieval Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we aim to show that an end-to-end deep neural network unifying training and indexing stages can significantly improve the recall performance of traditional methods. |
Yujing Wang; Haonan Wang; Yingyan Hou; Ziming Miao; Shibin Wu; Hao Sun; Qi Chen; Yuqing Xia; Chengmin Chi; Guoshuai Zhao; Zheng Liu; Xing Xie; Hao Sun; Weiwei Deng; Qi Zhang; Mao Yang; |
| 138 | Defining and Characterizing Reward Gaming Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We formally define reward gaming as situations where optimizing a proxy can decrease the true reward, and provide examples and theoretical results. |
Joar Skalse; Nikolaus Howe; Dmitrii Krasheninnikov; David Krueger; |
| 139 | OpenXAI: Towards A Transparent Evaluation of Model Explanations Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce OpenXAI, a flexible and comprehensive open source ecosystem for evaluating, comparing, and benchmarking SOTA as well as any newly proposed explanation methods. |
Chirag Agarwal; Satyapriya Krishna; Eshika Saxena; Martin Pawelczyk; Nari Johnson; Isha Puri; Marinka Zitnik; Himabindu Lakkaraju; |
| 140 | Towards Understanding Grokking: An Effective Theory of Representation Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We aim to understand grokking, a phenomenon where models generalize long after overfitting their training set. |
Ziming Liu; Ouail Kitouni; Niklas S Nolte; Eric Michaud; Max Tegmark; Mike Williams; |
| 141 | CyCLIP: Cyclic Contrastive Language-Image Pretraining Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a framework for cyclic consistency in contrastive language-image pretraining |
Shashank Goel; Hritik Bansal; Sumit Bhatia; Ryan Rossi; Vishwa Vinay; Aditya Grover; |
| 142 | Multi-Granularity Cross-modal Alignment for Generalized Medical Visual Representation Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we present a novel Multi-Granularity Cross-modal Alignment (MGCA) framework for generalized medical visual representation learning by harnessing the naturally exhibited semantic correspondences between medical image and radiology reports at three different levels, i.e., pathological region-level, instance-level, and disease-level. |
Fuying Wang; Yuyin Zhou; Shujun WANG; Varut Vardhanabhuti; Lequan Yu; |
| 143 | Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methods Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we propose a unifying framework under the helm of spectral manifold learning. |
Randall Balestriero; Yann LeCun; |
| 144 | Diverse Weight Averaging for Out-of-Distribution Generalization Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose Diverse Weight Averaging (DiWA), a new WA strategy whose main motivation is to increase the functional diversity across averaged models. |
Alexandre Rame; Matthieu Kirchmeyer; Thibaud Rahier; Alain Rakotomamonjy; Patrick Gallinari; Matthieu Cord; |
| 145 | TransTab: Learning Transferable Tabular Transformers Across Tables Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We are the first to fulfill pretraining, transfer learning, feature incremental learning, and zero-shot predictions across tabular datasets based on transferable tabular transformers (TransTab). |
Zifeng Wang; Jimeng Sun; |
| 146 | TAP-Vid: A Benchmark for Tracking Any Point in A Video Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we first formalize the problem, naming it tracking any point (TAP). We introduce a companion benchmark,TAP-Vid, which is composed of both real-world videos with accurate human annotations of point tracks, and synthetic videos with perfect ground-truth point tracks. |
Carl Doersch; Ankush Gupta; Larisa Markeeva; Adria Recasens; Lucas Smaira; Yusuf Aytar; Joao Carreira; Andrew Zisserman; Yi Yang; |
| 147 | Language Models with Image Descriptors Are Strong Few-Shot Video-Language Learners Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: The goal of this work is to build flexible video-language models that can generalize to various video-to-text tasks from few examples. |
Zhenhailong Wang; Manling Li; Ruochen Xu; Luowei Zhou; Jie Lei; Xudong Lin; Shuohang Wang; Ziyi Yang; Chenguang Zhu; Derek Hoiem; Shih-Fu Chang; Mohit Bansal; Heng Ji; |
| 148 | Attracting and Dispersing: A Simple Approach for Source-free Domain Adaptation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a simple but effective source-free domain adaptation (SFDA) method. |
Shiqi Yang; yaxing wang; kai wang; Shangling Jui; Joost van de Weijer; |
| 149 | Understanding and Extending Subgraph GNNs By Rethinking Their Symmetries Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we study the most prominent form of subgraph methods, which employs node-based subgraph selection policies such as ego-networks or node marking and deletion. |
Fabrizio Frasca; Beatrice Bevilacqua; Michael Bronstein; Haggai Maron; |
| 150 | Deep Model Reassembly Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we explore a novel knowledge-transfer task, termed as Deep Model Reassembly (DeRy), for general-purpose model reuse. |
Xingyi Yang; Daquan Zhou; Songhua Liu; Jingwen Ye; Xinchao Wang; |
| 151 | Model-Based Imitation Learning for Urban Driving Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present MILE: a Model-based Imitation LEarning approach for autonomous driving that scales to the complexity of urban driving scenes. |
Anthony Hu; Gianluca Corrado; Nicolas Griffiths; Zachary Murez; Corina Gurau; Hudson Yeo; Alex Kendall; Roberto Cipolla; Jamie Shotton; |
| 152 | Hidden Progress in Deep Learning: SGD Learns Parities Near The Computational Limit Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: While there are some accounts of how these resources modulate statistical capacity, far less is known about their effect on the computational problem of model training. This work conducts such an exploration through the lens of learning $k$-sparse parities of $n$ bits, a canonical family of problems which pose theoretical computational barriers. |
Boaz Barak; Benjamin Edelman; Surbhi Goel; Sham Kakade; Eran Malach; Cyril Zhang; |
| 153 | Coarse-to-Fine Vision-Language Pre-training with Fusion in The Backbone Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: However, most existing end-to-end pre-training approaches either only aim to tackle VL tasks such as image-text retrieval, visual question answering (VQA) and image captioning that test high-level understanding of images, or only target region-level understanding for tasks such as phrase grounding and object detection. We present FIBER (Fusion-In-the-Backbone-based transformER), a new VL model architecture that can seamlessly handle both these types of tasks. |
Zi-Yi Dou; Aishwarya Kamath; Zhe Gan; Pengchuan Zhang; Jianfeng Wang; Linjie Li; Zicheng Liu; Ce Liu; Yann LeCun; Nanyun Peng; Jianfeng Gao; Lijuan Wang; |
| 154 | SegViT: Semantic Segmentation with Plain Vision Transformers Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Specifically, we propose the Attention-to-Mask (ATM) module, in which the similarity maps between a set of learnable class tokens and the spatial feature maps are transferred to the segmentation masks. |
Bowen Zhang; Zhi Tian; Quan Tang; Xiangxiang Chu; Xiaolin Wei; Chunhua Shen; Yifan liu; |
| 155 | ConvMAE: Masked Convolution Meets Masked Autoencoders Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, our ConvMAE framework demonstrates that multi-scale hybrid convolution-transformer can learn more discriminative representations via the mask auto-encoding scheme. |
Peng Gao; Teli Ma; Hongsheng Li; Ziyi Lin; Jifeng Dai; Yu Qiao; |
| 156 | FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settings Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose a novel cross-silo dataset suite focused on healthcare, FLamby (Federated Learning AMple Benchmark of Your cross-silo strategies), to bridge the gap between theory and practice of cross-silo FL.FLamby encompasses 7 healthcare datasets with natural splits, covering multiple tasks, modalities, and data volumes, each accompanied with baseline training code. |
Jean Ogier du Terrail; Samy-Safwan Ayed; Edwige Cyffers; Felix Grimberg; Chaoyang He; Regis Loeb; Paul Mangold; Tanguy Marchand; Othmane Marfoq; Erum Mushtaq; Boris Muzellec; Constantin Philippenko; Santiago Silva; Maria Teleńczuk; Shadi Albarqouni; Salman Avestimehr; Aurélien Bellet; Aymeric Dieuleveut; Martin Jaggi; Sai Praneeth Karimireddy; Marco Lorenzi; Giovanni Neglia; Marc Tommasi; Mathieu Andreux; |
| 157 | Convergence for Score-based Generative Modeling with Polynomial Complexity Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We give the first fully polynomial convergence guarantees for score-based generative models. |
Holden Lee; Jianfeng Lu; Yixin Tan; |
| 158 | Self-Supervised Aggregation of Diverse Experts for Test-Agnostic Long-Tailed Recognition Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we study a more practical yet challenging task, called test-agnostic long-tailed recognition, where the training class distribution is long-tailed while the test class distribution is agnostic and not necessarily uniform. |
Yifan Zhang; Bryan Hooi; Lanqing Hong; Jiashi Feng; |
| 159 | HyperTree Proof Search for Neural Theorem Proving Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose an online training procedure for a transformer-based automated theorem prover. |
Guillaume Lample; Timothee Lacroix; Marie-Anne Lachaux; Aurelien Rodriguez; Amaury Hayat; Thibaut Lavril; Gabriel Ebner; Xavier Martinet; |
| 160 | Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratch Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We develop a new hidden trigger attack, Sleeper Agent, which employs gradient matching, data selection, and target model re-training during the crafting process. |
Hossein Souri; Liam Fowl; Rama Chellappa; Micah Goldblum; Tom Goldstein; |
| 161 | Improved Surface Reconstruction Using High-frequency Details Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose a novel method to improve the quality of surface reconstruction in neural rendering. |
Yiqun Wang; Ivan Skorokhodov; Peter Wonka; |
| 162 | High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves The Representation Related Papers Related Patents Related Grants Related Venues Related Experts View Abstract: We study the first gradient descent step on the first-layer parameters $\boldsymbol{W}$ in a two-layer neural network: $f(\boldsymbol{x}) = … |
Jimmy Ba; Murat Erdogdu; Taiji Suzuki; Zhichao Wang; Denny Wu; Greg Yang; |
| 163 | Cross Aggregation Transformer for Image Restoration Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: However, these methods lack direct interaction among different windows, which limits the establishment of long-range dependencies. To address the above issue, we propose a new image restoration model, Cross Aggregation Transformer (CAT). |
Zheng Chen; Yulun Zhang; Jinjin Gu; yongbing zhang; Linghe Kong; Xin Yuan; |
| 164 | A Continuous Time Framework for Discrete Denoising Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We provide the first complete continuous time framework for denoising diffusion models of discrete data. |
Andrew Campbell; Joe Benton; Valentin De Bortoli; Thomas Rainforth; George Deligiannidis; Arnaud Doucet; |
| 165 | RAMBO-RL: Robust Adversarial Model-Based Offline Reinforcement Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we present Robust Adversarial Model-Based Offline RL (RAMBO), a novel approach to model-based offline RL. |
Marc Rigter; Bruno Lacerda; Nick Hawes; |
| 166 | Sample Efficiency Matters: A Benchmark for Practical Molecular Optimization Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper thoroughly investigates the performance of 25 molecular design algorithms on 23 single-objective (scalar) optimization tasks with a particular focus on sample efficiency. |
Wenhao Gao; Tianfan Fu; Jimeng Sun; Connor Coley; |
| 167 | Weakly Supervised Causal Representation Learning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We show that causal factors and their causal structure can be identified from low-level data (e.g. pixels) observed before and after interventions. |
Johann Brehmer; Pim de Haan; Phillip Lippe; Taco Cohen; |
| 168 | Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videos Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose STEVE, an unsupervised model for object-centric learning in videos. |
Gautam Singh; Yi-Fu Wu; Sungjin Ahn; |
| 169 | SemMAE: Semantic-Guided Masking for Learning Masked Autoencoders Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we explore a potential visual analogue of words, i.e., semantic parts, and we integrate semantic information into the training process of MAE by proposing a Semantic-Guided Masking strategy. |
Gang Li; Heliang Zheng; Daqing Liu; Chaoyue Wang; Bing Su; Changwen Zheng; |
| 170 | VICRegL: Self-Supervised Learning of Local Visual Features Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: A new method called VICRegL is proposed that learns good global and local features simultaneously, yielding excellent performance on detection and segmentation tasks while maintaining good performance on classification tasks. |
Adrien Bardes; Jean Ponce; Yann LeCun; |
| 171 | Generative Neural Articulated Radiance Fields Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: These 3D GANs, however, have not been demonstrated for human bodies and the generated radiance fields of existing frameworks are not directly editable, limiting their applicability in downstream tasks. We propose a solution to these challenges by developing a 3D GAN framework that learns to generate radiance fields of human bodies or faces in a canonical pose and warp them using an explicit deformation field into a desired body pose or facial expression. |
Alexander Bergman; Petr Kellnhofer; Wang Yifan; Eric Chan; David Lindell; Gordon Wetzstein; |
| 172 | DeVRF: Fast Deformable Voxel Radiance Fields for Dynamic Scenes Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we present DeVRF, a novel representation to accelerate learning dynamic radiance fields. |
Jia-Wei Liu; Yan-Pei Cao; Weijia Mao; Wenqiao Zhang; David Junhao Zhang; Jussi Keppo; Ying Shan; Xiaohu Qie; Mike Zheng Shou; |
| 173 | Is Out-of-distribution Detection Learnable? Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: To study the generalization of OOD detection, in this paper, we investigate the probably approximately correct (PAC) learning theory of OOD detection, which is proposed by researchers as an open problem. |
Zhen Fang; Yixuan Li; Jie Lu; Jiahua Dong; Bo Han; Feng Liu; |
| 174 | On Feature Learning in The Presence of Spurious Correlations Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We explore the quality of representations learned by standard ERM training and specialized group robustness methods in the presence of spurious correlations. |
Pavel Izmailov; Polina Kirichenko; Nate Gruver; Andrew Wilson; |
| 175 | GMMSeg: Gaussian Mixture Based Generative Semantic Segmentation Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Though straightforward, this de facto paradigm neglects the underlying data distribution p(pixel feature|class), and struggles to identify out-of-distribution data. Going beyond this, we propose GMMSeg, a new family of segmentation models that rely on a dense generative classifier for the joint distribution p(pixel feature,class). |
Chen Liang; Wenguan Wang; Jiaxu Miao; Yi Yang; |
| 176 | GOOD: A Graph Out-of-Distribution Benchmark Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we aim at developing an OOD benchmark, known as GOOD, for graphs specifically. |
Shurui Gui; Xiner Li; Limei Wang; Shuiwang Ji; |
| 177 | Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: By plugging HST into DAUF, we establish the first Transformer-based deep unfolding method, Degradation-Aware Unfolding Half-Shuffle Transformer (DAUHST), for HSI reconstruction. |
Yuanhao Cai; Jing Lin; Haoqian Wang; Xin Yuan; Henghui Ding; Yulun Zhang; Radu Timofte; Luc V Gool; |
| 178 | Diffusion-based Molecule Generation with Informative Prior Bridges Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose a simple and novel approach to steer the training of diffusion-based generative models with physical and statistics prior information. |
Lemeng Wu; Chengyue Gong; Xingchao Liu; Mao Ye; Qiang Liu; |
| 179 | DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we utilize the strong alignment of textual and visual features pretrained with millions of auxiliary image-text pairs and propose \textit{Dual Context Optimization} (DualCoOp) as a unified framework for partial-label MLR and zero-shot MLR. |
Ximeng Sun; Ping Hu; Kate Saenko; |
| 180 | Learning Operators with Coupled Attention Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a novel operator learning method, LOCA (Learning Operators with Coupled Attention), motivated from the recent success of the attention mechanism. |
Georgios Kissas; Jacob Seidman; Leonardo Ferreira Guilhoto; Victor M. Preciado; George J. Pappas; Paris Perdikaris; |
| 181 | How Powerful Are K-hop Message Passing Graph Neural Networks Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we first theoretically analyze the expressive power and the limitation of K-hop message passing graph neural networks. Then, we propose a novel method to improve the K-hop message passing framework. |
Jiarui Feng; Yixin Chen; Fuhai Li; Anindya Sarkar; Muhan Zhang; |
| 182 | GENIE: Higher-Order Denoising Diffusion Solvers Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose Higher-Order Denoising Diffusion Solvers (GENIE): Based on truncated Taylor methods, we derive a novel higher-order solver that significantly accelerates synthesis. |
Tim Dockhorn; Arash Vahdat; Karsten Kreis; |
| 183 | CARD: Classification and Regression Diffusion Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we introduce classification and regression diffusion (CARD) models, which combine a denoising diffusion-based conditional generative model and a pre-trained conditional mean estimator, to accurately predict the distribution of y given x. |
Xizewen Han; Huangjie Zheng; Mingyuan Zhou; |
| 184 | Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a bimanual dexterous manipulation benchmark according to literature from cognitive science for comprehensive reinforcement learning research. |
Yuanpei Chen; Tianhao Wu; Shengjie Wang; Xidong Feng; Jiechuan Jiang; Zongqing Lu; Stephen McAleer; Hao Dong; Song-Chun Zhu; Yaodong Yang; |
| 185 | DENSE: Data-Free One-Shot Federated Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Despite the low communication cost, existing one-shot FL methods are mostly impractical or face inherent limitations, \eg a public dataset is required, %poor performance of the global model, clients’ models are homogeneous, and additional data/model information need to be uploaded. To overcome these issues, we propose a novel two-stage \textbf{D}ata-fre\textbf{E} o\textbf{N}e-\textbf{S}hot federated l\textbf{E}arning (DENSE) framework, which trains the global model by a data generation stage and a model distillation stage. |
Jie Zhang; Chen Chen; Bo Li; Lingjuan Lyu; Shuang Wu; Shouhong Ding; Chunhua Shen; Chao Wu; |
| 186 | Mildly Conservative Q-Learning for Offline Reinforcement Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper explores mild but enough conservatism for offline learning while not harming generalization. We propose Mildly Conservative Q-learning (MCQ), where OOD actions are actively trained by assigning them proper pseudo Q values. |
Jiafei Lyu; Xiaoteng Ma; Xiu Li; Zongqing Lu; |
| 187 | Two-Stream Network for Sign Language Recognition and Translation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To learn more meaningful representations and incorporate domain knowledge, such as handshape and facial expressions, we introduce a dual visual encoder containing two separate streams to model both the raw videos and the keypoint sequences generated by an off-the-shelf keypoint estimator. |
Yutong Chen; Ronglai Zuo; Fangyun Wei; Yu Wu; Shujie LIU; Brian Mak; |
| 188 | K-Radar: 4D Radar Object Detection for Autonomous Driving in Various Weather Conditions Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we introduce KAIST-Radar (K-Radar), a novel large-scale object detection dataset and benchmark that contains 35K frames of 4D Radar tensor (4DRT) data with power measurements along the Doppler, range, azimuth, and elevation dimensions, together with carefully annotated 3D bounding box labels of objects on the roads. |
Dong-Hee Paek; SEUNG-HYUN KONG; Kevin Tirta Wijaya; |
| 189 | Brain Network Transformer Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we study Transformer-based models for brain network analysis. |
Xuan Kan; Wei Dai; Hejie Cui; Zilong Zhang; Ying Guo; Carl Yang; |
| 190 | Squeezeformer: An Efficient Transformer for Automatic Speech Recognition Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: After reexamining the design choices for both the macro and micro-architecture of Conformer, we propose Squeezeformer which consistently outperforms the state-of-the-art ASR models under the same training schemes. |
Amir Gholami; Kurt Keutzer; Sehoon Kim; Nicholas Lee; Michael Mahoney; Jitendra Malik; Karttikeya Mangalam; Albert Shaw; |
| 191 | Preservation of The Global Knowledge By Not-True Distillation in Federated Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper suggests the forgetting global knowledge in federated learning, and proposes distillation-based algorithms to relieve it. |
Gihun Lee; Minchan Jeong; Yongjin Shin; Sangmin Bae; Se-Young Yun; |
| 192 | What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methods Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: However, it is not yet known: (1) how useful these methods are in real-world scenarios and (2) how well theoretical measures predict the usefulness of these methods for practical use by a human. To fill this gap, we conducted human psychophysics experiments at scale to evaluate the ability of human participants (n=1,150) to leverage representative attribution methods to predicting the decision of different image classifiers. |
Julien Colin; Thomas FEL; Remi Cadene; Thomas Serre; |
| 193 | On The Representation Collapse of Sparse Mixture of Experts Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose to estimate the routing scores between tokens and experts on a low-dimensional hypersphere. |
Zewen Chi; Li Dong; Shaohan Huang; Damai Dai; Shuming Ma; Barun Patra; Saksham Singhal; Payal Bajaj; XIA SONG; Xian-Ling Mao; Heyan Huang; Furu Wei; |
| 194 | Revisiting Neural Scaling Laws in Language and Vision Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To predict the benefit of scale empirically, we argue for a more rigorous methodology based on the extrapolation loss, instead of reporting the best-fitting (interpolating) parameters. |
Ibrahim Alabdulmohsin; Behnam Neyshabur; Xiaohua Zhai; |
| 195 | The Effects of Regularization and Data Augmentation Are Class Dependent Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this study, we demonstrate that techniques such as DA or weight decay produce a model with a reduced complexity that is unfair across classes. |
Randall Balestriero; Leon Bottou; Yann LeCun; |
| 196 | Rethinking and Scaling Up Graph Contrastive Learning: An Extremely Efficient Approach with Group Discrimination Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Inspired by an observation of a technical defect (i.e., inappropriate usage of Sigmoid function) commonly used in two representative GCL works, DGI and MVGRL, we revisit GCL and introduce a new learning paradigm for self-supervised graph representation learning, namely, Group Discrimination (GD), and propose a novel GD-based method called Graph Group Discrimination (GGD). |
YIZHEN ZHENG; Shirui Pan; Vincent CS Lee; Yu Zheng; Philip S Yu; |
| 197 | Remember The Past: Distilling Datasets Into Addressable Memories for Neural Networks Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose an algorithm that compresses the critical information of a large dataset into compact addressable memories. |
Zhiwei Deng; Olga Russakovsky; |
| 198 | Recurrent Memory Transformer Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose and study a memory-augmented segment-level recurrent Transformer (RMT). |
Aydar Bulatov; Yury Kuratov; Mikhail Burtsev; |
| 199 | PyramidCLIP: Hierarchical Feature Alignment for Vision-language Model Pretraining Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, in real scenarios, this assumption can be difficult to hold: the text description, obtained by crawling the affiliated metadata of the image, often suffers from the semantic mismatch and the mutual compatibility. To address these issues, we introduce PyramidCLIP, which constructs an input pyramid with different semantic levels for each modality, and aligns visual elements and linguistic elements in the form of hierarchy via peer-level semantics alignment and cross-level relation alignment. |
Yuting Gao; Jinfeng Liu; Zihan Xu; Jun Zhang; Ke Li; Rongrong Ji; Chunhua Shen; |
| 200 | The Privacy Onion Effect: Memorization Is Relative Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We demonstrate and analyze an Onion Effect of memorization: removing the "layer" of outlier points that are most vulnerable to a privacy attack exposes a new layer of previously-safe points to the same attack. |
Nicholas Carlini; Matthew Jagielski; Chiyuan Zhang; Nicolas Papernot; Andreas Terzis; Florian Tramer; |
| 201 | Block-Recurrent Transformers Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce the Block-Recurrent Transformer, which applies a transformer layer in a recurrent fashion along a sequence, and has linear complexity with respect to sequence length. |
DeLesley Hutchins; Imanol Schlag; Ethan Dyer; Behnam Neyshabur; Yuhuai Wu; |
| 202 | Generating Long Videos of Dynamic Scenes Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present a video generation model that accurately reproduces object motion, changes in camera viewpoint, and new content that arises over time. |
Tim Brooks; Janne Hellsten; Miika Aittala; Ting-Chun Wang; Timo Aila; Jaakko Lehtinen; Ming-Yu Liu; Alexei Efros; Tero Karras; |
| 203 | Pile of Law: Learning Responsible Data Filtering from The Law and A 256GB Open-Source Legal Dataset Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work we have examine how the law and legal data can inform data filtering practices and provide an extensive 256GB legal dataset (the Pile of Law) that can be used to learn these norms, and for pretraining. |
Peter Henderson; Mark Krass; Lucia Zheng; Neel Guha; Christopher D Manning; Dan Jurafsky; Daniel Ho; |
| 204 | Efficient Dataset Distillation Using Random Feature Approximation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Today’s best performing algorithm, \textit{Kernel Inducing Points} (KIP), which makes use of the correspondence between infinite-width neural networks and kernel-ridge regression, is prohibitively slow due to the exact computation of the neural tangent kernel matrix, scaling $O(|S|^2)$, with $|S|$ being the coreset size. To improve this, we propose a novel algorithm that uses a random feature approximation (RFA) of the Neural Network Gaussian Process (NNGP) kernel which reduces the kernel matrix computation to $O(|S|)$. |
Noel Loo; Ramin Hasani; Alexander Amini; Daniela Rus; |
| 205 | Quality Not Quantity: On The Interaction Between Dataset Design and Robustness of CLIP Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we introduce a testbed of six publicly available data sources—YFCC, LAION, Conceptual Captions, WIT, RedCaps, Shutterstock—to investigate how pre-training distributions induce robustness in CLIP. |
Thao Nguyen; Gabriel Ilharco; Mitchell Wortsman; Sewoong Oh; Ludwig Schmidt; |
| 206 | Compressible-composable NeRF Via Rank-residual Decomposition Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: However, its implicit representation causes difficulty in manipulating the models like the explicit mesh representation.Several recent advances in NeRF manipulation are usually restricted by a shared renderer network, or suffer from large model size. To circumvent the hurdle, in this paper, we present a neural field representation that enables efficient and convenient manipulation of models.To achieve this goal, we learn a hybrid tensor rank decomposition of the scene without neural networks. |
Jiaxiang Tang; Xiaokang Chen; Jingbo Wang; Gang Zeng; |
| 207 | Toward A Realistic Model of Speech Processing in The Brain with Self-supervised Learning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Focusing on the issue of speech processing, we here hypothesize that self-supervised algorithms trained on the raw waveform constitute a promising candidate. |
Juliette MILLET; Charlotte Caucheteux; pierre orhan; Yves Boubenec; Alexandre Gramfort; Ewan Dunbar; Christophe Pallier; Jean-Remi King; |
| 208 | VCT: A Video Compression Transformer Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We show how transformers can be used to vastly simplify neural video compression. |
Fabian Mentzer; George D Toderici; David Minnen; Sergi Caelles; Sung Jin Hwang; Mario Lucic; Eirikur Agustsson; |
| 209 | PEER: A Comprehensive and Multi-Task Benchmark for Protein Sequence Understanding Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This work proposes a comprehensive and multi-task benchmark for protein sequence understanding, which studies both single-task and multi-task learning. |
Minghao Xu; Zuobai Zhang; Jiarui Lu; Zhaocheng Zhu; Yangtian Zhang; Ma Chang; Runcheng Liu; Jian Tang; |
| 210 | Towards Better Evaluation for Dynamic Link Prediction Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper we proposed tools to improve evaluation of dynamic link prediction including new datasets, new negative sampling strategies, and a strong baseline. |
Farimah Poursafaei; Shenyang Huang; Kellin Pelrine; Reihaneh Rabbany; |
| 211 | Untargeted Backdoor Watermark: Towards Harmless and Stealthy Dataset Copyright Protection Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We explore how to design the untargeted backdoor watermark and how to use it for harmless and stealthy dataset copyright protection. |
Yiming Li; Yang Bai; Yong Jiang; Yong Yang; Shu-Tao Xia; Bo Li; |
| 212 | Convolutional Neural Networks on Graphs with Chebyshev Approximation, Revisited Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we revisit the problem of approximating the spectral graph convolutions with Chebyshev polynomials. |
Mingguo He; Zhewei Wei; Ji-Rong Wen; |
| 213 | Generative Time Series Forecasting with Diffusion, Denoise and Disentanglement Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we propose to address the time series forecasting problem with generative modeling. |
Yan Li; Xinjiang Lu; Yaqing Wang; Dejing Dou; |
| 214 | Object Scene Representation Transformer Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose Object Scene Representation Transformer (OSRT), a highly efficient 3D-centric model in which individual object representations naturally emerge through novel view synthesis. |
Mehdi S. M. Sajjadi; Daniel Duckworth; Aravindh Mahendran; Sjoerd van Steenkiste; Filip Pavetić; Mario Lucic; Leonidas Guibas; Klaus Greff; Thomas Kipf; |
| 215 | Sharpness-Aware Training for Free Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose Sharpness-Aware Training for Free, or SAF, which mitigates the sharp landscape at almost zero additional computational cost over the base optimizer. |
JIAWEI DU; Daquan Zhou; Joey Tianyi Zhou; Jiashi Feng; Vincent Tan; |
| 216 | General Cutting Planes for Bound-Propagation-Based Neural Network Verification Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we generalize the bound propagation procedure to allow the addition of arbitrary cutting plane constraints, including those involving relaxed integer variables that do not appear in existing bound propagation formulations. |
Huan Zhang; Shiqi Wang; Kaidi Xu; Linyi Li; Bo Li; Suman Jana; Cho-Jui Hsieh; J. Zico Kolter; |
| 217 | Thor: Wielding Hammers to Integrate Language Models and Automated Theorem Provers Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This presents a challenge for all theorem provers, especially the ones based on language models, due to their relative inability to reason over huge volumes of premises in text form. This paper introduces Thor, a framework integrating language models and automated theorem provers to overcome this difficulty. |
Albert Qiaochu Jiang; Wenda Li; Szymon Tworkowski; Konrad Czechowski; Tomasz Odrzygóźdź; Piotr Miłoś; Yuhuai Wu; Mateja Jamnik; |
| 218 | If Influence Functions Are The Answer, Then What Is The Question? Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: While influence estimates align well with leave-one-out retraining for linear models, recent works have shown this alignment is often poor in neural networks. In this work, we investigate the specific factors that cause this discrepancy by decomposing it into five separate terms. |
Juhan Bae; Nathan Ng; Alston Lo; Marzyeh Ghassemi; Roger Grosse; |
| 219 | ReCo: Retrieve and Co-segment for Zero-shot Transfer Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a new framework for zero-shot transfer semantic segmentation, which retrieves a set of unlabelled images of a concept using a language-image pre-trained model and co-segments the category regions using modern image representations. |
Gyungin Shin; Weidi Xie; Samuel Albanie; |
| 220 | Statistically Meaningful Approximation: A Case Study on Approximating Turing Machines with Transformers Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose a new notion of "statistically meaningful" approximation and show that neural nets can statistically-meaningfully approximate Boolean circuits and Turing machines. |
Colin Wei; Yining Chen; Tengyu Ma; |
| 221 | Capturing Failures of Large Language Models Via Human Cognitive Biases Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Large language models generate complex, open-ended outputs: instead of outputting a class label they write summaries, generate dialogue, or produce working code. In order to asses the reliability of these open-ended generation systems, we aim to identify qualitative categories of erroneous behavior, beyond identifying individual errors. |
Erik Jones; Jacob Steinhardt; |
| 222 | Which Explanation Should I Choose? A Function Approximation Perspective to Characterizing Post Hoc Explanations Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce a no free lunch theorem for explanation methods which demonstrates that no single method can perform optimally across all neighbourhoods and calls for choosing among methods. |
Tessa Han; Suraj Srinivas; Himabindu Lakkaraju; |
| 223 | HUMANISE: Language-conditioned Human Motion Generation in 3D Scenes Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To fill in the gap, we propose a large-scale and semantic-rich synthetic HSI dataset, denoted as HUMANISE, by aligning the captured human motion sequences with various 3D indoor scenes. |
Zan Wang; Yixin Chen; Tengyu Liu; Yixin Zhu; Wei Liang; Siyuan Huang; |
| 224 | K-LITE: Learning Transferable Visual Models with External Knowledge Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: K-LITE provides the first strong evidence that external knowledge benefits large-scale task-level visual transfer in image classification and object detection |
Sheng Shen; Chunyuan Li; Xiaowei Hu; Yujia Xie; Jianwei Yang; Pengchuan Zhang; Zhe Gan; Lijuan Wang; Lu Yuan; Ce Liu; Kurt Keutzer; Trevor Darrell; Anna Rohrbach; Jianfeng Gao; |
| 225 | Not Too Little, Not Too Much: A Theoretical Analysis of Graph (over)smoothing Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we consider simplified linear GNNs, and rigorously analyze two examples for which a finite number of mean aggregation steps provably improves the learning performance, before oversmoothing kisks in. |
Nicolas Keriven; |
| 226 | Deep Hierarchical Planning from Pixels Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce Director, a practical method for learning hierarchical behaviors directly from pixels by planning inside the latent space of a learned world model. |
Danijar Hafner; Kuang-Huei Lee; Ian Fischer; Pieter Abbeel; |
| 227 | ComENet: Towards Complete and Efficient Message Passing for 3D Molecular Graphs Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Notably, we propose the important rotation angles to fulfill global completeness. |
Limei Wang; Yi Liu; Yuchao Lin; Haoran Liu; Shuiwang Ji; |
| 228 | EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce VISOR, a new dataset of pixel annotations and a benchmark suite for segmenting hands and active objects in egocentric video. |
Ahmad Darkhalil; Dandan Shan; Bin Zhu; Jian Ma; Amlan Kar; Richard Higgins; Sanja Fidler; David Fouhey; Dima Damen; |
| 229 | Fully Sparse 3D Object Detection Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To enable efficient long-range LiDAR-based object detection, we build a fully sparse 3D object detector (FSD). |
Lue Fan; Feng Wang; Naiyan Wang; ZHAO-XIANG ZHANG; |
| 230 | RTFormer: Efficient Design for Real-Time Semantic Segmentation with Transformer Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose RTFormer, an efficient transformer for real-time semantic segmenation, which achieves better trade-off between performance and efficiency than CNN-based models. |
Jian Wang; Chenhui Gou; Qiman Wu; Haocheng Feng; Junyu Han; Errui Ding; Jingdong Wang; |
| 231 | Mind Reader: Reconstructing Complex Images from Brain Activities Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Unlike previous works that reconstruct images with single objects or simple shapes, our work aims to reconstruct image stimuli that are rich in semantics, closer to everyday scenes, and can reveal more perspectives. |
Sikun Lin; Thomas Sprague; Ambuj K Singh; |
| 232 | VITA: Video Instance Segmentation Via Object Token Association Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce a novel paradigm for offline Video Instance Segmentation (VIS), based on the hypothesis that explicit object-oriented information can be a strong clue for understanding the context of the entire sequence. |
Miran Heo; Sukjun Hwang; Seoung Wug Oh; Joon-Young Lee; Seon Joo Kim; |
| 233 | BOND: Benchmarking Unsupervised Outlier Node Detection on Static Attributed Graphs Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present BOND, a comprehensive benchmark for unsupervised node outlier detection on attributed static graphs. |
Kay Liu; Yingtong Dou; Yue Zhao; Xueying Ding; Xiyang Hu; Ruitong Zhang; Kaize Ding; Canyu Chen; Hao Peng; Kai Shu; Lichao Sun; Jundong Li; George H Chen; Zhihao Jia; Philip S Yu; |
| 234 | Learning Substructure Invariance for Out-of-Distribution Molecular Representations Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We aim to solve the out-of-distribution problem on molecule representation learning tasks from a substructure invariance perspective. |
Nianzu Yang; Kaipeng Zeng; Qitian Wu; Xiaosong Jia; Junchi Yan; |
| 235 | Decentralized Training of Foundation Models in Heterogeneous Environments Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We explore how to deploy the training of large-scale foundation models in a decentralized environment. |
Binhang Yuan; Yongjun He; Tianyi Zhang; Jared Davis; Tri Dao; Beidi Chen; Percy Liang; Christopher Ré; Ce Zhang; |
| 236 | Fully Convolutional One-Stage 3D Object Detection on LiDAR Range Images Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We present a simple yet effective fully convolutional one-stage 3D object detector for LiDAR point clouds of autonomous driving scenes, termed FCOS-LiDAR. |
Zhi Tian; Xiangxiang Chu; Xiaoming Wang; Xiaolin Wei; Chunhua Shen; |
| 237 | 3DILG: Irregular Latent Grids for 3D Generative Modeling Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a new representation for encoding 3D shapes as neural fields. |
Biao Zhang; Matthias Niessner; Peter Wonka; |
| 238 | Temporal Effective Batch Normalization in Spiking Neural Networks Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: To this end, we propose an effective normalization method called temporal effective batch normalization (TEBN). |
Chaoteng Duan; Jianhao Ding; Shiyan Chen; Zhaofei Yu; Tiejun Huang; |
| 239 | Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformer Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, through extensive empirical analysis, we first identify the bottleneck for severe performance drop comes from the information distortion of the low-bit quantized self-attention map. We then develop an information rectification module (IRM) and a distribution guided distillation (DGD) scheme for fully quantized vision transformers (Q-ViT) to effectively eliminate such distortion, leading to a fully quantized ViTs. |
Yanjing Li; Sheng Xu; Baochang Zhang; Xianbin Cao; Peng Gao; Guodong Guo; |
| 240 | Agreement-on-the-line: Predicting The Performance of Neural Networks Under Distribution Shift Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we show a similar surprising phenomena also holds for the agreement between pairs of neural network classifiers: whenever accuracy-on-the-line holds, we observe that the OOD agreement between the predictions of any two pairs of neural networks (with potentially different architectures) also observes a strong linear correlation with their ID agreement. |
Christina Baek; Yiding Jiang; Aditi Raghunathan; J. Zico Kolter; |
| 241 | Periodic Graph Transformers for Crystal Material Property Prediction Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose a transformer architecture, known as Matformer, for periodic graph representation learning. |
Keqiang Yan; Yi Liu; Yuchao Lin; Shuiwang Ji; |
| 242 | FedSR: A Simple and Effective Domain Generalization Method for Federated Learning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose a domain generalization learning method suitable for federated learning by implicit representation alignment |
A. Tuan Nguyen; Ser Nam Lim; Philip Torr; |
| 243 | SoundSpaces 2.0: A Simulation Platform for Visual-Acoustic Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We are releasing SoundSpaces 2.0: a fast, continuous, configurable and generalizable audio-visual simulation platform for visual acoustic machine learning research, e.g., audio-visual navigation, far-field speech recognition, and acoustic matching. |
Changan Chen; Carl Schissler; Sanchit Garg; Philip Kobernik; Alexander Clegg; Paul Calamia; Dhruv Batra; Philip Robinson; Kristen Grauman; |
| 244 | Hierarchical Graph Transformer with Adaptive Node Sampling Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we identify the main deficiencies of current graph transformers: (1) Existing node sampling strategies in Graph Transformers are agnostic to the graph characteristics and the training process. |
ZAIXI ZHANG; Qi Liu; Qingyong Hu; Chee-Kong Lee; |
| 245 | SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image Collections Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a joint optimization framework to estimate the shape, BRDF, and per-image camera pose and illumination. |
Mark Boss; Andreas Engelhardt; Abhishek Kar; Yuanzhen Li; Deqing Sun; Jonathan Barron; Hendrik PA Lensch; Varun Jampani; |
| 246 | Debiasing Graph Neural Networks Via Learning Disentangled Causal Substructure Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: By analyzing this problem in a causal view, we find that disentangling and decorrelating the causal and bias latent variables from the biased graphs are both crucial for debiasing. Inspiring by this, we propose a general disentangled GNN framework to learn the causal substructure and bias substructure, respectively. |
Shaohua Fan; Xiao Wang; Yanhu Mo; Chuan Shi; Jian Tang; |
| 247 | Wukong: A 100 Million Large-scale Chinese Cross-modal Pre-training Benchmark Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: A large-scale Chinese cross-modal dataset, called Wukong, containing 100 million image-text pairs is released. Models with either global similarity or token-wise similarity are pre-trained and benchmarked on extensive downstream tasks. |
Jiaxi Gu; Xiaojun Meng; Guansong Lu; Lu Hou; Niu Minzhe; Xiaodan Liang; Lewei Yao; Runhui Huang; Wei Zhang; Xin Jiang; Chunjing XU; Hang Xu; |
| 248 | NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: NeoRL presents conservative datasets for offline RL, highlights the complete pipeline for deploying offline RL in real-world applications, and also benchmarks recent offline RL algorithms on NeoRL under the complete pipeline. |
Rong-Jun Qin; Xingyuan Zhang; Songyi Gao; Xiong-Hui Chen; Zewen Li; Weinan Zhang; Yang Yu; |
| 249 | Debiased Self-Training for Semi-Supervised Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We tackle the bias issue in SSL by (1) decoupling the generation and utilization of pseudo labels; (2) estimating the worst case of pseudo labeling and optimizing the representation to avoid the worst case. |
Baixu Chen; Junguang Jiang; Ximei Wang; Pengfei Wan; Jianmin Wang; Mingsheng Long; |
| 250 | P2P: Tuning Pre-trained Image Models for Point Cloud Analysis with Point-to-Pixel Prompting Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: However, it is non-trivial to promote such a pretraining-tuning paradigm to the 3D vision, given the limited training data that are relatively inconvenient to collect. In this paper, we propose a new perspective of leveraging pre-trained 2D knowledge in 3D domain to tackle this problem, tuning pre-trained image models with the novel Point-to-Pixel prompting for point cloud analysis. |
Ziyi Wang; Xumin Yu; Yongming Rao; Jie Zhou; Jiwen Lu; |
| 251 | Benign Overfitting in Two-layer Convolutional Neural Networks Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we study the benign overfitting phenomenon in training a two-layer convolutional neural network (CNN). |
Yuan Cao; Zixiang Chen; Misha Belkin; Quanquan Gu; |
| 252 | ShapeCrafter: A Recursive Text-Conditioned 3D Shape Generation Model Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We recursively generate 3D shape distributions from progressively evolving phrase sequences. |
Rao Fu; Xiao Zhan; YIWEN CHEN; Daniel Ritchie; Srinath Sridhar; |
| 253 | PolarMix: A General Data Augmentation Technique for LiDAR Point Clouds Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper presents PolarMix, a point cloud augmentation technique that is simple and generic but can mitigate the data constraint effectively across various perception tasks and scenarios. |
Aoran Xiao; Jiaxing Huang; Dayan Guan; Kaiwen Cui; Shijian Lu; Ling Shao; |
| 254 | Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We study the asynchronous stochastic gradient descent algorithm, for distributed training over $n$ workers that might be heterogeneous. |
Anastasiia Koloskova; Sebastian Stich; Martin Jaggi; |
| 255 | DAGMA: Learning DAGs Via M-matrices and A Log-Determinant Acyclicity Characterization Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose a \emph{fundamentally different} acyclicity characterization based on the log-determinant (log-det) function, which leverages the nilpotency property of DAGs. |
Kevin Bello; Bryon Aragam; Pradeep Ravikumar; |
| 256 | TwiBot-22: Towards Graph-Based Twitter Bot Detection Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We make the case for graph-based Twitter bot detection and propose a graph-based benchmark TwiBot-22, which addresses the issues of limited dataset scale, incomplete graph structure, and low annotation quality in previous datasets. |
Shangbin Feng; Zhaoxuan Tan; Herun Wan; Ningnan Wang; Zilong Chen; Binchi Zhang; Qinghua Zheng; Wenqian Zhang; Zhenyu Lei; Shujie Yang; Xinshun Feng; Qingyue Zhang; Hongrui Wang; Yuhan Liu; Yuyang Bai; Heng Wang; Zijian Cai; Yanbo Wang; Lijing Zheng; Zihan Ma; Jundong Li; Minnan Luo; |
| 257 | In Defense of The Unitary Scalarization for Deep Multi-Task Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We show that a basic multi-task learning optimizer performs on par with specialized algorithms and suggest a possible explanation based on regularization. |
Vitaly Kurin; Alessandro De Palma; Ilya Kostrikov; Shimon Whiteson; Pawan K Mudigonda; |
| 258 | Boosting The Transferability of Adversarial Attacks with Reverse Adversarial Perturbation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Deep neural networks (DNNs) have been shown to be vulnerable to adversarial examples, which can produce erroneous predictions by injecting imperceptible perturbations. In this work, we study the transferability of adversarial examples, which is significant due to its threat to real-world applications where model architecture or parameters are usually unknown. |
Zeyu Qin; Yanbo Fan; Yi Liu; Li Shen; Yong Zhang; Jue Wang; Baoyuan Wu; |
| 259 | Poisson Flow Generative Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a new "Poisson flow" generative model~(PFGM) that maps a uniform distribution on a high-dimensional hemisphere into any data distribution. |
Yilun Xu; Ziming Liu; Max Tegmark; Tommi Jaakkola; |
| 260 | Riemannian Diffusion Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose a continuous-time diffusion model for data represented on a Riemannian manifold. |
Chin-Wei Huang; Milad Aghajohari; Joey Bose; Prakash Panangaden; Aaron Courville; |
| 261 | Neural Temporal Walks: Motif-Aware Representation Learning on Continuous-Time Dynamic Graphs Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we introduce a novel method, Neural Temporal Walks (NeurTWs), for representation learning on continuous-time dynamic graphs. |
Ming Jin; Yuan-Fang Li; Shirui Pan; |
| 262 | Iron: Private Inference on Transformers Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Our main contribution is to provide several new secure protocols for matrix multiplication and complex non-linear functions like Softmax, GELU activations, and LayerNorm, which are critical components of Transformers. |
Meng Hao; Hongwei Li; Hanxiao Chen; Pengzhi Xing; Guowen Xu; Tianwei Zhang; |
| 263 | The Dollar Street Dataset: Images Representing The Geographic and Socioeconomic Diversity of The World Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We present Dollar Street, a supervised dataset that contains 38,479 images of everyday household items from homes around the world, including tags for objects and demographic data such as region, country and home monthly income. |
William Gaviria Rojas; Sudnya Diamos; Keertan Kini; David Kanter; Vijay Janapa Reddi; Cody Coleman; |
| 264 | DGraph: A Large-Scale Financial Dataset for Graph Anomaly Detection Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper present DGraph, a real-world dynamic graph in the finance domain. |
Xuanwen Huang; Yang Yang; Yang Wang; Chunping Wang; Zhisheng Zhang; Jiarong Xu; Lei Chen; Michalis Vazirgiannis; |
| 265 | MinVIS: A Minimal Video Instance Segmentation Framework Without Video-based Training Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose MinVIS, a minimal video instance segmentation (VIS) framework that achieves state-of-the-art VIS performance with neither video-based architectures nor training procedures. |
De-An Huang; Zhiding Yu; Anima Anandkumar; |
| 266 | FP8 Quantization: The Power of The Exponent Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: However, low-bit floating point numbers have an extra degree of freedom, assigning some bits to work on an exponential scale instead. This paper exhaustively investigates this benefit of the floating point format for neural network inference. |
Andrey Kuzmin; Mart van Baalen; Yuwei Ren; Markus Nagel; Jorn Peters; Tijmen Blankevoort; |
| 267 | Assaying Out-Of-Distribution Generalization in Transfer Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we take a unified view of previous work, highlighting message discrepancies that we address empirically, and providing recommendations on how to measure the robustness of a model and how to improve it. |
Florian Wenzel; Andrea Dittadi; Peter Gehler; Carl-Johann Simon-Gabriel; Max Horn; Dominik Zietlow; David Kernert; Chris Russell; Thomas Brox; Bernt Schiele; Bernhard Schölkopf; Francesco Locatello; |
| 268 | CATER: Intellectual Property Protection on Text Generation APIs Via Conditional Watermarks Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a novel Conditional wATERmarking framework (CATER) for protecting the IP right of text generation APIs caused by imitation attacks. |
Xuanli He; Qiongkai Xu; Yi Zeng; Lingjuan Lyu; Fangzhao Wu; Jiwei Li; Ruoxi Jia; |
| 269 | DIMES: A Differentiable Meta Solver for Combinatorial Optimization Problems Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper addresses the scalability challenge in large-scale combinatorial optimization by proposing a novel approach, namely, DIMES. |
Ruizhong Qiu; Zhiqing Sun; Yiming Yang; |
| 270 | GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networks Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose GLIF, a unified spiking neuron, to fuse different bio-features in different neuronal behaviors, enlarging the representation space of spiking neurons. |
Xingting Yao; Fanrong Li; Zitao Mo; Jian Cheng; |
| 271 | Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We analyze feature learning in infinite-width neural networks trained with gradient flow through a self-consistent dynamical field theory. |
Blake Bordelon; Cengiz Pehlevan; |
| 272 | Recovering Private Text in Federated Learning of Language Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we present a novel attack method FILM (Federated Inversion attack for Language Models) for federated learning of language models—for the first time, we show the feasibility of recovering text from large batch sizes of up to 128 sentences. |
Samyak Gupta; Yangsibo Huang; Zexuan Zhong; Tianyu Gao; Kai Li; Danqi Chen; |
| 273 | Semantic Probabilistic Layers for Neuro-Symbolic Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We design a predictive layer for structured-output prediction (SOP) that can be plugged into any neural network guaranteeing its predictions are consistent with a set of predefined symbolic constraints. |
Kareem Ahmed; Stefano Teso; Kai-Wei Chang; Guy Van den Broeck; Antonio Vergari; |
| 274 | Sym-NCO: Leveraging Symmetricity for Neural Combinatorial Optimization Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper presents a novel training scheme, Sym-NCO, which is a regularizer-based training scheme that leverages universal symmetricities in various CO problems and solutions. |
Minsu Kim; Junyoung Park; Jinkyoo Park; |
| 275 | Discovered Policy Optimisation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper we explore the Mirror Learning space by meta-learning a “drift” function. |
Chris Lu; Jakub Kuba; Alistair Letcher; Luke Metz; Christian Schroeder de Witt; Jakob Foerster; |
| 276 | RORL: Robust Offline Reinforcement Learning Via Conservative Smoothing Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To trade off robustness and conservatism, we propose Robust Offline Reinforcement Learning (RORL) with a novel conservative smoothing technique. |
Rui Yang; Chenjia Bai; Xiaoteng Ma; Zhaoran Wang; Chongjie Zhang; Lei Han; |
| 277 | BOME! Bilevel Optimization Made Easy: A Simple First-Order Approach Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose a simple first-order BO algorithm that depends only on first-order gradient information, requires no implicit differentiation, and is practical and efficient for large-scale non-convex functions in deep learning. |
Mao Ye; Bo Liu; Stephen Wright; Peter Stone; Qiang Liu; |
| 278 | M4Singer: A Multi-Style, Multi-Singer and Musical Score Provided Mandarin Singing Corpus Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: The lack of publicly available high-quality and accurately labeled datasets has long been a major bottleneck for singing voice synthesis (SVS). To tackle this problem, we present M4Singer, a free-to-use Multi-style, Multi-singer Mandarin singing collection with elaborately annotated Musical scores as well as its benchmarks. |
Lichao Zhang; Ruiqi Li; Shoutong Wang; Liqun Deng; Jinglin Liu; Yi Ren; Jinzheng He; Rongjie Huang; Jieming Zhu; Xiao Chen; Zhou Zhao; |
| 279 | Wild-Time: A Benchmark of In-the-Wild Distribution Shift Over Time Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To address this gap, we curate Wild-Time, a benchmark of 7 datasets that reflect temporal distribution shifts arising in a variety of real-world applications, including drug discovery, patient prognosis, and news classification. On these datasets, we systematically benchmark 13 approaches with various inductive biases. |
Huaxiu Yao; Caroline Choi; Bochuan Cao; Yoonho Lee; Pang Wei Koh; Chelsea Finn; |
| 280 | ShuffleMixer: An Efficient ConvNet for Image Super-Resolution Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a simple and effective approach, ShuffleMixer, for lightweight image super-resolution that combines large convolution and channel split-shuffle operation. |
Long Sun; Jinshan Pan; Jinhui Tang; |
| 281 | Theseus: A Library for Differentiable Nonlinear Optimization Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present Theseus, an efficient application-agnostic open source library for differentiable nonlinear least squares (DNLS) optimization built on PyTorch, providing a common framework for end-to-end structured learning in robotics and vision. |
Luis Pineda; Taosha Fan; Maurizio Monge; Shobha Venkataraman; Paloma Sodhi; Ricky T. Q. Chen; Joseph Ortiz; Daniel DeTone; Austin Wang; Stuart Anderson; Jing Dong; Brandon Amos; Mustafa Mukadam; |
| 282 | Self-Supervised Visual Representation Learning with Semantic Grouping Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Instead, we propose contrastive learning from data-driven semantic slots, namely SlotCon, for joint semantic grouping and representation learning. |
Xin Wen; Bingchen Zhao; Anlin Zheng; Xiangyu Zhang; Xiaojuan Qi; |
| 283 | Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approach Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose an efficient and effective training scheme coined as Sparse SAM (SSAM), which achieves sparse perturbation by a binary mask. |
Peng Mi; Li Shen; Tianhe Ren; Yiyi Zhou; Xiaoshuai Sun; Rongrong Ji; Dacheng Tao; |
| 284 | Picking on The Same Person: Does Algorithmic Monoculture Lead to Outcome Homogenization? Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: While sharing offers advantages like amortizing effort, it also has risks. We introduce and formalize one such risk, $\textit{outcome homogenization}$, defined here as the extent to which particular individuals or groups experience the same outcomes across different deployments. |
Rishi Bommasani; Kathleen A. Creel; Ananya Kumar; Dan Jurafsky; Percy Liang; |
| 285 | OnePose++: Keypoint-Free One-Shot Object Pose Estimation Without CAD Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose an keypoint-free one-shot object pose estimation method that handles low-textured objects without knowing CAD models. |
Xingyi He; Jiaming Sun; Yuang Wang; Di Huang; Hujun Bao; Xiaowei Zhou; |
| 286 | Effective Backdoor Defense By Exploiting Sensitivity of Poisoned Samples Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Given a backdoored model, we observe that the feature representations of poisoned samples with trigger are more sensitive to transformations than those of clean samples. |
Weixin Chen; Baoyuan Wu; Haoqian Wang; |
| 287 | Learning Neural Acoustic Fields Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: While recent advances in learned implicit functions have led to increasingly higher quality representations of the visual world, there have not been commensurate advances in learning spatial auditory representations. To address this gap, we introduce Neural Acoustic Fields (NAFs), an implicit representation that captures how sounds propagate in a physical scene. |
Andrew Luo; Yilun Du; Michael Tarr; Josh Tenenbaum; Antonio Torralba; Chuang Gan; |
| 288 | A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarks Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To address these issues, we categorize existing works into three practical scenarios in which attackers release datasets, pre-trained models, and fine-tuned models respectively, then discuss their unique evaluation methodologies. |
Ganqu Cui; Lifan Yuan; Bingxiang He; Yangyi Chen; Zhiyuan Liu; Maosong Sun; |
| 289 | Understanding The Generalization Benefit of Normalization Layers: Sharpness Reduction Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Motivated by the long-held belief that flatter minima lead to better generalization, this paper gives mathematical analysis and supporting experiments suggesting that normalization (together with accompanying weight-decay) encourages GD to reduce the sharpness of loss surface. |
Kaifeng Lyu; Zhiyuan Li; Sanjeev Arora; |
| 290 | Sequencer: Deep LSTM for Image Classification Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Against this background, there is growing interest in what inductive bias is suitable for computer vision. Here we propose Sequencer, a novel and competitive architecture alternative to ViT that provides a new perspective on these issues. |
Yuki Tatsunami; Masato Taki; |
| 291 | Local Latent Space Bayesian Optimization Over Structured Inputs Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose LOL-BO, which adapts the notion of trust regions explored in recent work on high-dimensional Bayesian optimization to the structured setting. |
Natalie Maus; Haydn Jones; Juston Moore; Matt Kusner; John Bradshaw; Jacob Gardner; |
| 292 | DC-BENCH: Dataset Condensation Benchmark Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This work provides the first large-scale standardized benchmark on Dataset Condensation. |
Justin CUI; Ruochen Wang; Si Si; Cho-Jui Hsieh; |
| 293 | IM-Loss: Information Maximization Loss for Spiking Neural Networks Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, the forward-passing $0/1$ spike quantization will cause information loss and accuracy degradation. To deal with this problem, the Information maximization loss (IM-Loss) that aims at maximizing the information flow in the SNN is proposed in the paper. |
Yufei Guo; Yuanpei Chen; Liwen Zhang; Xiaode Liu; Yinglei Wang; Xuhui Huang; Zhe Ma; |
| 294 | Semantic Exploration from Language Abstractions and Pretrained Representations Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Novelty-based exploration methods can suffer in high-dimensional state spaces, such as continuous partially-observable 3D environments. We address this challenge by defining novelty using semantically meaningful state abstractions, which can be found in learned representations shaped by natural language. |
Allison Tam; Neil Rabinowitz; Andrew Lampinen; Nicholas Roy; Stephanie Chan; DJ Strouse; Jane Wang; Andrea Banino; Felix Hill; |
| 295 | UViM: A Unified Modeling Approach for Vision with Learned Guiding Codes Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a unified model for computer vision, which does not require any task-specific components. |
Alexander Kolesnikov; André Susano Pinto; Lucas Beyer; Xiaohua Zhai; Jeremiah Harmsen; Neil Houlsby; |
| 296 | Robust Reinforcement Learning Using Offline Data Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose a robust RL algorithm called Robust Fitted Q-Iteration (RFQI), which uses only an offline dataset to learn the optimal robust policy. |
Kishan Panaganti; Zaiyan Xu; Dileep Kalathil; Mohammad Ghavamzadeh; |
| 297 | FedAvg with Fine Tuning: Local Updates Lead to Representation Learning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This surprising performance of such a simple method, however, is not fully understood from a theoretical point of view. In this paper, we formally investigate this phenomenon in the multi-task linear representation setting. |
Liam Collins; Hamed Hassani; Aryan Mokhtari; Sanjay Shakkottai; |
| 298 | Fine-Grained Semantically Aligned Vision-Language Pre-Training Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we introduce LOUPE, a fine-grained semantically aLigned visiOn-langUage PrE-training framework, which learns fine-grained semantic alignment from the novel perspective of game-theoretic interactions. |
Juncheng Li; XIN HE; Longhui Wei; Long Qian; Linchao Zhu; Lingxi Xie; Yueting Zhuang; Qi Tian; Siliang Tang; |
| 299 | CAGroup3D: Class-Aware Grouping for 3D Object Detection on Point Clouds Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present a novel two-stage fully sparse convolutional 3D object detection framework, named CAGroup3D. |
Haiyang Wang; lihe Ding; Shaocong Dong; Shaoshuai Shi; Aoxue Li; Jianan Li; Zhenguo Li; Liwei Wang; |
| 300 | CLiMB: A Continual Learning Benchmark for Vision-and-Language Tasks Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper presents CLiMB, a benchmark to study the challenge of learning vision-language tasks in a continual learning setting, and to systematically evaluate how upstream continual learning can rapidly transfer to new multi- and unimodal tasks. |
Tejas Srinivasan; Ting-Yun Chang; Leticia Pinto Alva; Georgios Chochlakis; Mohammad Rostami; Jesse Thomason; |
| 301 | Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Hence, we propose Pre-trained Image Encoder for Generalizable visual reinforcement learning (PIE-G), a simple yet effective framework that can generalize to the unseen visual scenarios in a zero-shot manner. |
Zhecheng Yuan; Zhengrong Xue; Bo Yuan; Xueqian Wang; YI WU; Yang Gao; Huazhe Xu; |
| 302 | SemiFL: Semi-Supervised Federated Learning for Unlabeled Clients with Alternate Training Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose SemiFL to address the problem of combining communication efficient FL like FedAvg with Semi-Supervised Learning (SSL). |
Enmao Diao; Jie Ding; Vahid Tarokh; |
| 303 | REVIVE: Regional Visual Representation Matters in Knowledge-Based Visual Question Answering Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Specifically, we observe in most state-of-the-art knowledge-based VQA methods: 1) visual features are extracted either from the whole image or in a sliding window manner for retrieving knowledge, and the important relationship within/among object regions is neglected; 2) visual features are not well utilized in the final answering model, which is counter-intuitive to some extent. Based on these observations, we propose a new knowledge-based VQA method REVIVE, which tries to utilize the explicit information of object regions not only in the knowledge retrieval stage but also in the answering model. |
Yuanze Lin; Yujia Xie; Dongdong Chen; Yichong Xu; Chenguang Zhu; Lu Yuan; |
| 304 | Why So Pessimistic? Estimating Uncertainties for Offline RL Through Ensembles, and Why Their Independence Matters Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Motivated by the success of ensembles for uncertainty estimation in supervised learning, we take a renewed look at how ensembles of $Q$-functions can be leveraged as the primary source of pessimism for offline reinforcement learning (RL). |
Seyed Kamyar Seyed Ghasemipour; Shixiang (Shane) Gu; Ofir Nachum; |
| 305 | Green Hierarchical Vision Transformer for Masked Image Modeling Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present an efficient approach for Masked Image Modeling (MIM) with hierarchical Vision Transformers (ViTs), e.g., Swin Transformer, allowing the hierarchical ViTs to discard masked patches and operate only on the visible ones. |
Lang Huang; Shan You; Mingkai Zheng; Fei Wang; Chen Qian; Toshihiko Yamasaki; |
| 306 | Uni-Perceiver-MoE: Learning Sparse Generalist Models with Conditional MoEs Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we find that interference among different tasks and modalities is the main factor to this phenomenon. To mitigate such interference, we introduce the Conditional Mixture of Experts (Conditional MoEs) to generalist models. |
Jinguo Zhu; Xizhou Zhu; Wenhai Wang; Xiaohua Wang; Hongsheng Li; Xiaogang Wang; Jifeng Dai; |
| 307 | Imbalance Trouble: Revisiting Neural-Collapse Geometry Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Here we thus ask whether it can be made invariant to class imbalances. Towards this end, we adopt the unconstrained feature model (UFM), a recent theoretical model for studying neural collapse, and introduce $\text{\emph{Simplex-Encoded-Labels Interpolation}}$ (SELI) as an invariant characterization of the neural collapse phenomenon. |
Christos Thrampoulidis; Ganesh Ramachandra Kini; Vala Vakilian; Tina Behnia; |
| 308 | Addressing Leakage in Concept Bottleneck Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Leakage adversarily affects the performance and interpretability of concept bottleneck models. We address the underlying causes. |
Marton Havasi; Sonali Parbhoo; Finale Doshi-Velez; |
| 309 | Advancing Model Pruning Via Bi-level Optimization Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Specifically, we formulate the pruning problem from a fresh and novel viewpoint, bi-level optimization (BLO). |
Yihua Zhang; Yuguang Yao; Parikshit Ram; pu zhao; Tianlong Chen; Mingyi Hong; Yanzhi Wang; Sijia Liu; |
| 310 | How Mask Matters: Towards Theoretical Understandings of Masked Autoencoders Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose a new theoretical understanding of how MAE works and why the choice of mask ratio is so important for MAE from a graph perspective. |
Qi Zhang; Yifei Wang; Yisen Wang; |
| 311 | BYOL-Explore: Exploration By Bootstrapped Prediction Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We present BYOL-Explore, a conceptually simple yet general approach for curiosity-driven exploration in visually complex environments. |
Zhaohan Guo; Shantanu Thakoor; Miruna Pislar; Bernardo Avila Pires; Florent Altché; Corentin Tallec; Alaa Saade; Daniele Calandriello; Jean-Bastien Grill; Yunhao Tang; Michal Valko; Remi Munos; Mohammad Gheshlaghi Azar; Bilal Piot; |
| 312 | Semi-supervised Semantic Segmentation with Prototype-based Consistency Regularization Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This diversity will make the label propagation hard from pixels to pixels. To address this problem, we propose a novel approach to regularize the distribution of within-class features to ease label propagation difficulty. |
Haiming Xu; Lingqiao Liu; Qiuchen Bian; Zhen Yang; |
| 313 | Do Current Multi-Task Optimization Methods in Deep Learning Even Help? Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We perform a large-scale study of the effects of specialized optimization methods for deep multi-task models. |
Derrick Xin; Behrooz Ghorbani; Justin Gilmer; Ankush Garg; Orhan Firat; |
| 314 | Supported Policy Optimization for Offline Reinforcement Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper presents Supported Policy OpTimization (SPOT), which is directly derived from the theoretical formalization of the density-based support constraint. |
Jialong Wu; Haixu Wu; Zihan Qiu; Jianmin Wang; Mingsheng Long; |
| 315 | Handcrafted Backdoors in Deep Neural Networks Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We argue that a supply-chain attacker has more attack techniques available by introducing a $handcrafted$ attack that directly manipulates a model’s weights. |
Sanghyun Hong; Nicholas Carlini; Alexey Kurakin; |
| 316 | Learning to Break The Loop: Analyzing and Mitigating Repetitions for Neural Text Generation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We analyze the consetutive sentence repetitions in language models and propose a simple and effective method to mitigate it. |
Jin Xu; Xiaojiang Liu; Jianhao Yan; Deng Cai; Huayang Li; Jian Li; |
| 317 | Learning Latent Seasonal-Trend Representations for Time Series Forecasting Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Motivated by the success of disentangled variational autoencoder in computer vision and classical time series decomposition, we plan to infer a couple of representations that depict seasonal and trend components of time series. To achieve this goal, we propose LaST, which, based on variational inference, aims to disentangle the seasonal-trend representations in the latent space. |
Zhiyuan Wang; Xovee Xu; Goce Trajcevski; Weifeng Zhang; Ting Zhong; Fan Zhou; |
| 318 | OST: Improving Generalization of DeepFake Detection Via One-Shot Test-Time Training Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we introduce a new learning paradigm specially designed for the generalizable deepfake detection task. |
Liang Chen; Yong Zhang; Yibing Song; Jue Wang; Lingqiao Liu; |
| 319 | Streaming Radiance Fields for 3D Video Synthesis Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present an explicit-grid based method for efficiently reconstructing streaming radiance fields for novel view synthesis of real world dynamic scenes. |
Lingzhi LI; Zhen Shen; zhongshu wang; Li Shen; Ping Tan; |
| 320 | Variational Inference Via Wasserstein Gradient Flows Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose principled methods for VI, in which $\hat \pi$ is taken to be a Gaussian or a mixture of Gaussians, which rest upon the theory of gradient flows on the Bures–Wasserstein space of Gaussian measures. |
Marc Lambert; Sinho Chewi; Francis Bach; Silvère Bonnabel; Philippe Rigollet; |
| 321 | NOMAD: Nonlinear Manifold Decoders for Operator Learning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Here we present NOMAD, a novel operator learning framework with a nonlinear decoder map capable of learning finite dimensional representations of nonlinear submanifolds in function spaces. |
Jacob Seidman; Georgios Kissas; Paris Perdikaris; George J. Pappas; |
| 322 | Does GNN Pretraining Help Molecular Representation? Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We investigate graph pretraining on molecular representation. We conduct thorough ablation studies on the key components of GNN pretraining, and found that many occasions the benefits from self-supervised pretraining on molecular data is negligible. |
Ruoxi Sun; Hanjun Dai; Adams Yu; |
| 323 | Diffusion Visual Counterfactual Explanations Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Current approaches for the generation of VCEs are restricted to adversarially robust models and often contain non-realistic artefacts or are restricted to image classification problems with few classes. In this paper we overcome this by generating Diffusion Visual Counterfactual Explanations (DVCEs) for arbitrary ImageNet classifiers via a diffusion process. |
Maximilian Augustin; Valentyn Boreiko; Francesco Croce; Matthias Hein; |
| 324 | Language Conditioned Spatial Relation Reasoning for 3D Object Grounding Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work we propose a language-conditioned transformer model for grounding 3D objects and their spatial relations. |
Shizhe Chen; Pierre-Louis Guhur; Makarand Tapaswi; Cordelia Schmid; Ivan Laptev; |
| 325 | Improving Intrinsic Exploration with Language Abstractions Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Instead, we explore natural language as a general medium for highlighting relevant abstractions in an environment. |
Jesse Mu; Victor Zhong; Roberta Raileanu; Minqi Jiang; Noah Goodman; Tim Rocktäschel; Edward Grefenstette; |
| 326 | Learning Single-index Models with Shallow Neural Networks Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we introduce a natural class of shallow neural networks and study its ability to learn single-index models via \textit{gradient descent}. |
Alberto Bietti; Joan Bruna; Clayton Sanford; Min Jae Song; |
| 327 | Rethinking Alignment in Video Super-Resolution Transformers Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Nevertheless, such designs will dramatically increase the computational burden, and cannot deal with large motions. Therefore, we propose a new and efficient alignment method called patch alignment, which aligns image patches instead of pixels. |
Shuwei Shi; Jinjin Gu; Liangbin Xie; Xintao Wang; Yujiu Yang; Chao Dong; |
| 328 | Non-deep Networks Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This begs the question — is it possible to build high-performing “non-deep neural networks? We show that it is. |
Ankit Goyal; Alexey Bochkovskiy; Jia Deng; Vladlen Koltun; |
| 329 | Divide and Contrast: Source-free Domain Adaptation Via Adaptive Contrastive Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we present Divide and Contrast (DaC), a new paradigm for SFUDA that strives to connect the good ends of both worlds while bypassing their limitations. |
Ziyi Zhang; Weikai Chen; Hui Cheng; Zhen Li; Siyuan Li; Liang Lin; Guanbin Li; |
| 330 | PFL-Bench: A Comprehensive Benchmark for Personalized Federated Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose the first comprehensive benchmark for personalized Federated Learning, containing more than 10 datasets, 20 pFL methods, and systematic evaluation with highlighted benefits and potential of pFL. |
Daoyuan Chen; Dawei Gao; Weirui Kuang; Yaliang Li; Bolin Ding; |
| 331 | EnvPool: A Highly Parallel Reinforcement Learning Environment Execution Engine Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we aim to address a common bottleneck in the RL training system, i.e., parallel environment execution, which is often the slowest part of the whole system but receives little attention. |
Jiayi Weng; Min Lin; Shengyi Huang; Bo Liu; Denys Makoviichuk; Viktor Makoviychuk; Zichen Liu; Yufan Song; Ting Luo; Yukun Jiang; Zhongwen Xu; Shuicheng Yan; |
| 332 | NaturalProver: Grounded Mathematical Proof Generation with Language Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We study large-scale language models on two new generation tasks: suggesting the next step in a mathematical proof, and full proof generation. |
Sean Welleck; Jiacheng Liu; Ximing Lu; Hannaneh Hajishirzi; Yejin Choi; |
| 333 | Adam Can Converge Without Any Modification On Update Rules Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We prove that, when the 2nd-order momentum parameter $\beta_2$ is large and 1st-order momentum parameter $\beta_1 < \sqrt{\beta_2}<1$, Adam converges to the neighborhood of critical points. |
Yushun Zhang; Congliang Chen; Naichen Shi; Ruoyu Sun; Zhi-Quan Luo; |
| 334 | Learning Efficient Vision Transformers Via Fine-Grained Manifold Distillation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we fully utilize the patch-level information and propose a fine-grained manifold distillation method for transformer-based networks. |
Zhiwei Hao; Jianyuan Guo; Ding Jia; Kai Han; Yehui Tang; Chao Zhang; Han Hu; Yunhe Wang; |
| 335 | Amortized Inference for Causal Structure Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose to amortize causal structure learning. |
Lars Lorch; Scott Sussex; Jonas Rothfuss; Andreas Krause; Bernhard Schölkopf; |
| 336 | Matryoshka Representation Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Our main contribution is Matryoshka Representation Learning (MRL) which encodes information at different granularities and allows a single embedding to adapt to the computational constraints of downstream tasks. |
Aditya Kusupati; Gantavya Bhatt; Aniket Rege; Matthew Wallingford; Aditya Sinha; Vivek Ramanujan; William Howard-Snyder; Kaifeng Chen; Sham Kakade; Prateek Jain; Ali Farhadi; |
| 337 | Exploring The Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we explore domain-adaptive training to reduce the toxicity of language models. |
Boxin Wang; Wei Ping; Chaowei Xiao; Peng Xu; Mostofa Patwary; Mohammad Shoeybi; Bo Li; Anima Anandkumar; Bryan Catanzaro; |
| 338 | Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progress Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Overall, this work argues for an alternative approach to RL research, which we believe could significantly improve real-world RL adoption and help democratize it further. |
Rishabh Agarwal; Max Schwarzer; Pablo Samuel Castro; Aaron Courville; Marc Bellemare; |
| 339 | When Does Return-conditioned Supervised Learning Work for Offline Reinforcement Learning? Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we provide a rigorous study of the capabilities and limitations of RCSL something which is crucially missing in previous work. |
David Brandfonbrener; Alberto Bietti; Jacob Buckman; Romain Laroche; Joan Bruna; |
| 340 | Graph Neural Networks Are Dynamic Programmers Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We use category theory and abstract algebra to further uncover the relationship between graph neural nets and dynamic programming, which was previously done handwavily over specific examples. |
Andrew J Dudzik; Petar Veličković; |
| 341 | Adversarial Training for High-stakes Reliability Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We used a safe language generation task (“avoid injuries”) as a testbed for achieving high reliability through adversarial training. |
Daniel Ziegler; Seraphina Nix; Lawrence Chan; Tim Bauman; Peter Schmidt-Nielsen; Tao Lin; Adam Scherlis; Noa Nabeshima; Benjamin Weinstein-Raun; Daniel de Haas; Buck Shlegeris; Nate Thomas; |
| 342 | Simulation-guided Beam Search for Neural Combinatorial Optimization Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose simulation-guided beam search method and its combination with EAS (efficient active search) that significantly improve inference performances of neural approaches for combinatorial optimization. |
Jinho Choo; Yeong-Dae Kwon; Jihoon Kim; Jeongwoo Jae; André Hottung; Kevin Tierney; Youngjune Gwon; |
| 343 | Object Representations As Fixed Points: Training Iterative Refinement Algorithms with Implicit Differentiation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This often requires the use iterative refinement procedures that break symmetries among equally plausible explanations for the data, but most prior works differentiate through the unrolled refinement process, which can make optimization exceptionally challenging. In this work, we observe that such iterative refinement methods can be made differentiable by means of the implicit function theorem, and develop an implicit differentiation approach that improves the stability and tractability of training such models by decoupling the forward and backward passes. |
Michael Chang; Tom Griffiths; Sergey Levine; |
| 344 | A Unified Framework for Deep Symbolic Regression Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose a strategy to integrate five disparate solution strategies for symbolic regression into a unified framework, resulting in a new state-of-the-art on SRBench benchmarks. |
Mikel Landajuela; Chak Shing Lee; Jiachen Yang; Ruben Glatt; Claudio P Santiago; Ignacio Aravena; Terrell Mundhenk; Garrett Mulcahy; Brenden K Petersen; |
| 345 | PKD: General Distillation Framework for Object Detectors Via Pearson Correlation Coefficient Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a general distillation framework for object detectors via Pearson Correlation Coefficient to focus on the relational information from the teacher. |
Weihan Cao; Jianfei Gao; Anda Cheng; Ke Cheng; Yifan Zhang; Jian Cheng; |
| 346 | MoVQ: Modulating Quantized Vectors for High-Fidelity Image Generation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Although two-stage Vector Quantized (VQ) generative models allow for synthesizing high-fidelity and high-resolution images, their quantization operator encodes similar patches within an image into the same index, resulting in a repeated artifact for similar adjacent regions using existing decoder architectures. To address this issue, we propose to incorporate the spatially conditional normalization to modulate the quantized vectors so as to insert spatially variant information to the embedded index maps, encouraging the decoder to generate more photorealistic images. |
Chuanxia Zheng; Tung-Long Vuong; Jianfei Cai; Dinh Phung; |
| 347 | GlanceNets: Interpretabile, Leak-proof Concept-based Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce a new class of self-explainable models based on interpretable concepts. |
Emanuele Marconato; Andrea Passerini; Stefano Teso; |
| 348 | Adversarial Style Augmentation for Domain Generalized Urban-Scene Segmentation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a novel adversarial style augmentation approach for domain generalization in semantic segmentation, which is easy to implement and can effectively improve the model performance on unseen real domains. |
Zhun Zhong; Yuyang Zhao; Gim Hee Lee; Nicu Sebe; |
| 349 | Weakly Supervised Representation Learning with Sparse Perturbations Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a natural estimation procedure based on this theory and illustrate it on low-dimensional synthetic and image-based experiments. |
Kartik Ahuja; Jason Hartford; Yoshua Bengio; |
| 350 | A Comprehensive Study on Large-Scale Graph Training: Benchmarking and Rethinking Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present a comprehensive and fair benchmark study on large-scale graph training and further propose a new layer-wise training manner the achieves new SOTA performance on large-scale graph datasets. |
Keyu Duan; Zirui Liu; Peihao Wang; Wenqing Zheng; Kaixiong Zhou; Tianlong Chen; Xia Hu; Zhangyang Wang; |
| 351 | Tenrec: A Large-scale Multipurpose Benchmark Dataset for Recommender Systems Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we describe Tenrec, a novel and publicly available data collection for RS that records various user feedback from four different recommendation scenarios. |
guanghu yuan; Fajie Yuan; Yudong Li; Beibei Kong; Shujie Li; Lei Chen; Min Yang; Chenyun YU; Bo Hu; Zang Li; Yu Xu; Xiaohu Qie; |
| 352 | Signal Propagation in Transformers: Theoretical Perspectives and The Role of Rank Collapse Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: The question of if and how rank collapse affects training is still largely unanswered, and its investigation is necessary for a more comprehensive understanding of this architecture. In this work, we shed new light on the causes and the effects of this phenomenon. |
Sotiris Anagnostidis; Luca Biggio; Lorenzo Noci; Antonio Orvieto; Sidak Pal Singh; Aurelien Lucchi; |
| 353 | Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we introduce a Long-Form Video-Language Pre-training (LF-VLP) model, and train it on a large-scale long-form video and paragraph dataset constructed from an existing public dataset. |
Yuchong Sun; Bei Liu; Hongwei Xue; Ruihua Song; Huan Yang; Jianlong Fu; |
| 354 | Robustness to Unbounded Smoothness of Generalized SignSGD Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we show that clipping is not indispensable for Adam-type algorithms in tackling such scenarios: we theoretically prove that a generalized SignSGD algorithm can obtain similar convergence rates as SGD with clipping but does not need explicit clipping at all. |
Michael Crawshaw; Mingrui Liu; Francesco Orabona; Wei Zhang; Zhenxun Zhuang; |
| 355 | KERPLE: Kernelized Relative Positional Embedding for Length Extrapolation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose KERPLE, a framework that generalizes relative position embedding for extrapolation by kernelizing positional differences. |
Ta-Chung Chi; Ting-Han Fan; Peter J Ramadge; Alexander Rudnicky; |
| 356 | Probable Domain Generalization Via Quantile Risk Minimization Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose Quantile Risk Minimization for achieving *probable* domain generalization, where predictors are trained to generalize with a desired probability. |
Cian Eastwood; Alexander Robey; Shashank Singh; Julius von Kügelgen; Hamed Hassani; George J. Pappas; Bernhard Schölkopf; |
| 357 | Category-Level 6D Object Pose Estimation in The Wild: A Semi-Supervised Learning Approach and A New Dataset Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose a new model, called Rendering for Pose estimation network RePoNet), that is jointly trained using the free ground-truths with the synthetic data, and a silhouette matching objective function on the real-world data. |
Yang Fu; Xiaolong Wang; |
| 358 | BadPrompt: Backdoor Attacks on Continuous Prompts Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we conduct the first study on the vulnerability of the continuous prompt learning algorithm to backdoor attacks. |
Xiangrui Cai; Haidong Xu; Sihan Xu; Ying ZHANG; Yuan xiaojie; |
| 359 | Unpacking Reward Shaping: Understanding The Benefits of Reward Engineering on Sample Complexity Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we build on the framework of novelty-based exploration to provide a simple scheme for incorporating shaped rewards into RL along with an analysis tool to show that particular choices of reward shaping provably improve sample efficiency. |
Abhishek Gupta; Aldo Pacchiano; Yuexiang Zhai; Sham Kakade; Sergey Levine; |
| 360 | Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representations Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose Expectation-Maximization Contrastive Learning (EMCL) to learn compact video-and-language representations. |
Peng Jin; Fa Jin Huang; Fenglin Liu; Xian Wu; Shen Ge; Guoli Song; David Clifton; Jie Chen; |
| 361 | Feature-Proxy Transformer for Few-Shot Segmentation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper revives the straightforward framework of “feature extractor $+$ linear classification head” and proposes a novel Feature-Proxy Transformer (FPTrans) method, in which the “proxy” is the vector representing a semantic class in the linear classification head. |
Jian-Wei Zhang; Yifan Sun; Yi Yang; Wei Chen; |
| 362 | Towards Learning Universal Hyperparameter Optimizers with Transformers Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we introduce the OptFormer, the first text-based Transformer HPO framework that provides a universal end-to-end interface for jointly learning policy and function prediction when trained on vast tuning data from the wild, such as Google’s Vizier database, one of the world’s largest HPO datasets. |
Yutian Chen; Xingyou Song; Chansoo Lee; Zi Wang; Richard Zhang; David Dohan; Kazuya Kawakami; Greg Kochanski; Arnaud Doucet; Marc’Aurelio Ranzato; Sagi Perel; Nando de Freitas; |
| 363 | Privacy of Noisy Stochastic Gradient Descent: More Iterations Without More Privacy Loss Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We revisit the most commonly used algorithm for private convex optimization (Noisy SGD, aka SGLD), and establish a fundamental phenomenon: after a small burn-in period, running SGD longer leaks no additional privacy. |
Jason Altschuler; Kunal Talwar; |
| 364 | Supervised Training of Conditional Monge Maps Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To account for and incorporate that context in OT estimation, we introduce \textsc{CondOT}, an approach to estimate OT maps conditioned on a context variable, using several pairs of measures $(\mu_i, \nu_i)$ tagged with a context label~$c_i$. |
Charlotte Bunne; Andreas Krause; Marco Cuturi; |
| 365 | Towards Understanding The Mixture-of-Experts Layer in Deep Learning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we formally study how the MoE layer improves the performance of neural network learning and why the mixture model will not collapse into a single model. |
Zixiang Chen; Yihe Deng; Yue Wu; Quanquan Gu; Yuanzhi Li; |
| 366 | BinauralGrad: A Two-Stage Conditional Diffusion Probabilistic Model for Binaural Audio Synthesis Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we formulate the synthesis process from a different perspective by decomposing the binaural audio into a common part that shared by the left and right channels as well as a specific part that differs in each channel. Accordingly, we propose BinauralGrad, a novel two-stage framework equipped with diffusion models to synthesize them respectively. |
Yichong Leng; Zehua Chen; Junliang Guo; Haohe Liu; Jiawei Chen; Xu Tan; Danilo Mandic; Lei He; Xiangyang Li; Tao Qin; Sheng Zhao; Tie-Yan Liu; |
| 367 | Intermediate Prototype Mining Transformer for Few-Shot Semantic Segmentation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Specifically, we design an Intermediate Prototype Mining Transformer (IPMT) to learn the prototype in an iterative way. |
YUANWEI LIU; Nian Liu; Xiwen Yao; Junwei Han; |
| 368 | Learning Dynamical Systems Via Koopman Operator Regression in Reproducing Kernel Hilbert Spaces Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We study a class of dynamical systems modelled as stationary Markov chains that admit an invariant distribution via the corresponding transfer or Koopman operator. |
Pietro Novelli; Vladimir Kostic; Massimiliano Pontil; Andreas Maurer; Carlo Ciliberto; Lorenzo Rosasco; |
| 369 | All You Need Is A Good Functional Prior for Bayesian Deep Learning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This poses a challenge because modern neural networks are characterized by a large number of parameters, and the choice of these priors has an uncontrolled effect on the induced functional prior, which is the distribution of the functions obtained by sampling the parameters from their prior distribution. We argue that this is a hugely limiting aspect of Bayesian deep learning, and this work tackles this limitation in a practical and effective way. |
Ba-Hien Tran; Simone Rossi; Dimitrios Milios; Maurizio Filippone; |
| 370 | Riemannian Score-Based Generative Modelling Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce here \emph{Riemannian Score-based Generative Models} (RSGMs), a class of generative models extending SGMs to Riemannian manifolds. |
Valentin De Bortoli; Emile Mathieu; Michael Hutchinson; James Thornton; Yee Whye Teh; Arnaud Doucet; |
| 371 | Spherical Channels for Modeling Atomic Interactions Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose the Spherical Channel Network (SCN) to model atomic energies and forces. |
Larry Zitnick; Abhishek Das; Adeesh Kolluru; Janice Lan; Muhammed Shuaibi; Anuroop Sriram; Zachary Ulissi; Brandon Wood; |
| 372 | Ordered Subgraph Aggregation Networks Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Further, current approaches either use all subgraphs of a given size, sample them uniformly at random, or use hand-crafted heuristics to select them, oblivious to the given data distribution. Here, we offer a unified way to study such architectures by introducing a theoretical framework and extending the known expressivity results of subgraph-enhanced graph neural networks. |
Chendi Qian; Gaurav Rattan; Floris Geerts; Mathias Niepert; Christopher Morris; |
| 373 | Are All Losses Created Equal: A Neural Collapse Perspective Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: A broad family of loss functions leads to neural collapse solutions hence are equivalent on training set; moreover, they exhibit largely identical performance on test data as well. |
Jinxin Zhou; Chong You; Xiao Li; Kangning Liu; Sheng Liu; Qing Qu; Zhihui Zhu; |
| 374 | Gradient Flow Dynamics of Shallow ReLU Networks for Square Loss and Orthogonal Inputs Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This article presents, for orthogonal input vectors, a precise description of the gradient flow dynamics of training one-hidden layer ReLU neural networks for the mean squared error at small initialisation. |
Etienne Boursier; Loucas PILLAUD-VIVIEN; Nicolas Flammarion; |
| 375 | Unified Optimal Transport Framework for Universal Domain Adaptation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Moreover, they cannot recognize different categories among target-private samples as these private samples are treated as a whole. In this paper, we propose to use Optimal Transport (OT) to handle these issues under a unified framework, namely UniOT. |
Wanxing Chang; Ye Shi; Hoang Tuan; Jingya Wang; |
| 376 | Learning to Reconstruct Missing Data from Spatiotemporal Graphs with Sparse Observations Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In particular, we propose a novel class of attention-based architectures that, given a set of highly sparse discrete observations, learn a representation for points in time and space by exploiting a spatiotemporal diffusion architecture aligned with the imputation task. |
Ivan Marisca; Andrea Cini; Cesare Alippi; |
| 377 | Revisiting Graph Contrastive Learning from The Perspective of Graph Spectrum Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Then we theoretically prove that GCL is able to learn the invariance information by contrastive invariance theorem, together with our GAME rule, for the first time, we uncover that the learned representations by GCL essentially encode the low-frequency information, which explains why GCL works. Guided by this rule, we propose a spectral graph contrastive learning module (SpCo), which is a general and GCL-friendly plug-in. |
Nian Liu; Xiao Wang; Deyu Bo; Chuan Shi; Jian Pei; |
| 378 | A Policy-Guided Imitation Approach for Offline Reinforcement Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this study, we propose an alternative approach, inheriting the training stability of imitation-style methods while still allowing logical out-of-distribution generalization. |
Haoran Xu; Li Jiang; Li Jianxiong; Xianyuan Zhan; |
| 379 | Generic Bounds on The Approximation Error for Physics-informed (and) Operator Learning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose a very general framework for deriving rigorous bounds on the approximation error for physics-informed neural networks (PINNs) and operator learning architectures such as DeepONets and FNOs as well as for physics-informed operator learning. |
Tim De Ryck; Siddhartha Mishra; |
| 380 | Neural Surface Reconstruction of Dynamic Scenes with Monocular RGB-D Camera Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose Neural-DynamicReconstruction (NDR), a template-free method to recover high-fidelity geometry and motions of a dynamic scene from a monocular RGB-D camera. |
Hongrui Cai; Wanquan Feng; Xuetao Feng; Yan Wang; Juyong Zhang; |
| 381 | CageNeRF: Cage-based Neural Radiance Field for Generalized 3D Deformation and Animation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we propose a novel framework for deforming and animating the neural radiance field learned on arbitrary objects. |
Yicong Peng; Yichao Yan; Shengqi Liu; Yuhao Cheng; Shanyan Guan; Bowen Pan; Guangtao Zhai; Xiaokang Yang; |
| 382 | Singular Value Fine-tuning: Few-shot Segmentation Requires Few-parameters Fine-tuning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present a solution to overcome the overfitting problem, leading to better model generalization on learning novel classes. |
Yanpeng Sun; Qiang Chen; Xiangyu He; Zechao Li; Jian Wang; Haocheng Feng; Junyu Han; Errui Ding; Jian Cheng; Jingdong Wang; |
| 383 | Efficient and Effective Augmentation Strategy for Adversarial Training Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose an effective augmentation strategy for Adversarial Training that can be integrated with several Adversarial Training algorithms and data augmentations. |
Sravanti Addepalli; Samyak Jain; Venkatesh Babu R; |
| 384 | Delving Into Sequential Patches for Deepfake Detection Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Based on low-level temporal inconsistency understanding, we identify deepfake videos in a more robust and generalizable way with model designs in a Transfomer style. |
Jiazhi Guan; Hang Zhou; Zhibin Hong; Errui Ding; Jingdong Wang; Chengbin Quan; Youjian Zhao; |
| 385 | LAMP: Extracting Text from Gradients with Language Model Priors Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a novel attack for text reconstruction from gradients in federated learning based on language model priors. |
Mislav Balunovic; Dimitar Dimitrov; Nikola Jovanović; Martin Vechev; |
| 386 | Non-stationary Transformers: Rethinking The Stationarity in Time Series Forecasting Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: To tackle the dilemma between series predictability and model capability, we propose Non-stationary Transformers as a generic framework with two interdependent modules: Series Stationarization and De-stationary Attention. |
Yong Liu; Haixu Wu; Jianmin Wang; Mingsheng Long; |
| 387 | Root Cause Analysis of Failures in Microservices Through Causal Discovery Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose a scalable algorithm for quickly detecting the root cause of failure in complex microservice architectures. |
Azam Ikram; Sarthak Chakraborty; Subrata Mitra; Shiv Saini; Saurabh Bagchi; Murat Kocaoglu; |
| 388 | Online Training Through Time for Spiking Neural Networks Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose online training through time (OTTT) for SNNs, which is derived from BPTT to enable forward-in-time learning by tracking presynaptic activities and leveraging instantaneous loss and gradients. |
Mingqing Xiao; Qingyan Meng; Zongpeng Zhang; Di He; Zhouchen Lin; |
| 389 | Contrastive Adapters for Foundation Model Group Robustness Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We thus propose contrastive adapting, which contrastively trains adapters to bring sample embeddings close to both their ground-truth class embeddings and same-class sample embeddings. |
Michael Zhang; Christopher Ré; |
| 390 | Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Specifically, vanilla prompt learning may struggle to utilize atypical instances by rote during fully-supervised training or overfit shallow patterns with low-shot data. To alleviate such limitations, we develop RetroPrompt with the motivation of decoupling knowledge from memorization to help the model strike a balance between generalization and memorization. |
Xiang Chen; Lei Li; Ningyu Zhang; Xiaozhuan Liang; Shumin Deng; Chuanqi Tan; Fei Huang; Luo Si; Huajun Chen; |
| 391 | Dynamic Graph Neural Networks Under Spatio-Temporal Distribution Shift Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we propose to handle spatio-temporal distribution shifts in dynamic graphs by discovering and utilizing {\it invariant patterns}, i.e., structures and features whose predictive abilities are stable across distribution shifts, which faces two key challenges: 1) How to discover the complex variant and invariant spatio-temporal patterns in dynamic graphs, which involve both time-varying graph structures and node features. |
Zeyang Zhang; Xin Wang; Ziwei Zhang; Haoyang Li; Zhou Qin; Wenwu Zhu; |
| 392 | Semi-supervised Vision Transformers at Scale Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Our proposed method, dubbed Semi-ViT, achieves comparable or better performance than the CNN counterparts in the semi-supervised classification setting. |
Zhaowei Cai; Avinash Ravichandran; Paolo Favaro; Manchen Wang; Davide Modolo; Rahul Bhotika; Zhuowen Tu; Stefano Soatto; |
| 393 | FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we present an openly accessible FinRL-Meta library that has been actively maintained by the FinRL community. |
Xiao-Yang Liu; Ziyi Xia; Jingyang Rui; Jiechao Gao; Hongyang Yang; Ming Zhu; Christina Wang; Zhaoran Wang; Jian Guo; |
| 394 | Escaping from The Barren Plateau Via Gaussian Initializations in Deep Variational Quantum Circuits Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose a Gaussian initialization strategy addressing the vanishing gradient problem in variational quantum circuits with theoretical guarantees. |
Kaining Zhang; Liu Liu; Min-Hsiu Hsieh; Dacheng Tao; |
| 395 | Estimating Noise Transition Matrix with Label Correlations for Noisy Multi-Label Learning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose a new method to estimate the transition matrices by exploiting label correlations for noisy multi-label learning. |
Shikun Li; Xiaobo Xia; Hansong Zhang; Yibing Zhan; Shiming Ge; Tongliang Liu; |
| 396 | SparCL: Sparse Continual Learning on The Edge Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose a novel framework called Sparse Continual Learning (SparCL), which is the first study that leverages sparsity to enable cost-effective continual learning on edge devices. |
Zifeng Wang; Zheng Zhan; Yifan Gong; Geng Yuan; Wei Niu; Tong Jian; Bin Ren; Stratis Ioannidis; Yanzhi Wang; Jennifer Dy; |
| 397 | Improved Differential Privacy for SGD Via Optimal Private Linear Operators on Adaptive Streams Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We prove fundamental theoretical results on the applicability of matrix factorizations to the adaptive streaming setting, and provide a new parameter-free fixed-point algorithm for computing optimal factorizations. |
Sergey Denisov; H. Brendan McMahan; John Rush; Adam Smith; Abhradeep Guha Thakurta; |
| 398 | Incorporating Bias-aware Margins Into Contrastive Loss for Collaborative Filtering Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To reduce the negative impact of popularity bias on CF models, we incorporate Biasaware margins into Contrastive loss and propose a simple yet effective BC Loss, where the margin tailors quantitatively to the bias degree of each user-item interaction. |
An Zhang; Wenchang Ma; Xiang Wang; Tat-Seng Chua; |
| 399 | Asynchronous SGD Beats Minibatch SGD Under Arbitrary Delays Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We prove much better theoretical guarantees for asynchronous SGD, which depend on the number of workers rather than the delays. |
Blake Woodworth; Mathieu Even; Konstantin Mishchenko; Francis Bach; |
| 400 | No Free Lunch from Deep Learning in Neuroscience: A Case Study Through Models of The Entorhinal-Hippocampal Circuit Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: The central claims of recent deep learning-based models of brain circuits are that they make novel predictions about neural phenomena or shed light on the fundamental functions being optimized. We show, through the case-study of grid cells in the entorhinal-hippocampal circuit, that one may get neither. |
Rylan Schaeffer; Mikail Khona; Ila Fiete; |
| 401 | VLMbench: A Compositional Benchmark for Vision-and-Language Manipulation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: VLMbench is the first benchmark that compositional designs for vision-and-language reasoning and categorizes the manipulation tasks from the perspectives of task constraints. |
Kaizhi Zheng; Xiaotong Chen; Odest Chadwicke Jenkins; Xin Wang; |
| 402 | When Does Dough Become A Bagel? Analyzing The Remaining Mistakes on ImageNet Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We focus on the multi-label subset evaluation of ImageNet, where today’s best models achieve upwards of 97% top-1 accuracy. |
Vijay Vasudevan; Benjamin Caine; Raphael Gontijo Lopes; Sara Fridovich-Keil; Rebecca Roelofs; |
| 403 | Rethinking Generalization in Few-Shot Classification Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we take a closer look at the implications in the context of few-shot learning. |
Markus Hiller; Rongkai Ma; Mehrtash Harandi; Tom Drummond; |
| 404 | Accelerated Zeroth-Order and First-Order Momentum Methods from Mini to Minimax Optimization Related Papers Related Patents Related Grants Related Venues Related Experts View Abstract: In the paper, we propose a class of accelerated zeroth-order and first-order momentum methods for both nonconvex mini-optimization and minimax-optimization. Specifically, we … |
Feihu Huang; Shangqian Gao; Jian Pei; Heng Huang; |
| 405 | Generalization Analysis of Message Passing Neural Networks on Large Random Graphs Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We study the generalization error of MPNNs in graph classification and regression. |
Sohir Maskey; Ron Levie; Yunseok Lee; Gitta Kutyniok; |
| 406 | Deep Fourier Up-Sampling Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This is the first attempt to propose a theoretically feasible Deep Fourier Up-sampling for multi-scale modeling. |
man zhou; Hu Yu; Jie Huang; Feng Zhao; Jinwei Gu; Chen Change Loy; Deyu Meng; Chongyi Li; |
| 407 | Are Defenses for Graph Neural Networks Robust? Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Adaptive evaluation reveals that most examined adversarial defenses for GNNs show no or only marginal improvement in robustness |
Felix Mujkanovic; Simon Geisler; Aleksandar Bojchevski; Stephan Günnemann; |
| 408 | A Unified Analysis of Federated Learning with Arbitrary Client Participation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We present a unified framework for analyzing the convergence of federated learning with arbitrary participation of clients. |
Shiqiang Wang; Mingyue Ji; |
| 409 | Reinforcement Learning with Neural Radiance Fields Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We learn state representations of scenes using supervision from neural radiance fields, and show that using these in downstream reinforcement learning tasks improves sample efficiency. |
Danny Driess; Ingmar Schubert; Pete Florence; Yunzhu Li; Marc Toussaint; |
| 410 | Margin-Based Few-Shot Class-Incremental Learning with Class-Level Overfitting Mitigation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To mitigate class-level overfitting (CO) in margin-based classification for the few-shot class-incremental learning task, we first interpret CO from pattern learning, and then propose a method to mitigate CO and achieve SOTA performance. |
Yixiong Zou; Shanghang Zhang; Yuhua Li; Ruixuan Li; |
| 411 | Exploring The Whole Rashomon Set of Sparse Decision Trees Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We provide the first technique for completely enumerating the Rashomon set for sparse decision trees; in fact, our work provides the first complete enumeration of any Rashomon set for a non-trivial problem with a highly nonlinear discrete function class. |
Rui Xin; Chudi Zhong; Zhi Chen; Takuya Takagi; Margo Seltzer; Cynthia Rudin; |
| 412 | Concept Embedding Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose a novel concept-based interpretable architecture capable of learning meaningful concept embedding representations and supporting test-time concept interventions. |
Mateo Espinosa Zarlenga; Pietro Barbiero; Gabriele Ciravegna; Giuseppe Marra; Francesco Giannini; Michelangelo Diligenti; Zohreh Shams; Frederic Precioso; Stefano Melacci; Adrian Weller; Pietro Lió; Mateja Jamnik; |
| 413 | Manifold Interpolating Optimal-Transport Flows for Trajectory Inference Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We present a method called Manifold Interpolating Optimal-Transport Flow (MIOFlow) that learns stochastic, continuous population dynamics from static snapshot samples taken at sporadic timepoints. |
Guillaume Huguet; Daniel Sumner Magruder; Oluwadamilola Fasina; Alexander Tong; Manik Kuchroo; Guy Wolf; Smita Krishnaswamy; |
| 414 | CLEAR: Generative Counterfactual Explanations on Graphs Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we study the problem of counterfactual explanation generation on graphs. |
Jing Ma; Ruocheng Guo; Saumitra Mishra; Aidong Zhang; Jundong Li; |
| 415 | Contrastive Graph Structure Learning Via Information Bottleneck for Recommendation Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Here, we propose a Contrastive Graph Structure Learning via Information Bottleneck (CGI) for recommendation, which adaptively learns whether to drop an edge or node to obtain optimized graph structures in an end-to-end manner. |
Chunyu Wei; Jian Liang; Di Liu; Fei Wang; |
| 416 | Museformer: Transformer with Fine- and Coarse-Grained Attention for Music Generation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we propose Museformer, a Transformer with a novel fine- and coarse-grained attention for symbolic music generation. |
Botao Yu; Peiling Lu; Rui Wang; Wei Hu; Xu Tan; Wei Ye; Shikun Zhang; Tao Qin; Tie-Yan Liu; |
| 417 | On The Identifiability of Nonlinear ICA: Sparsity and Beyond Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We show the identifiability of nonlinear ICA with unconditional priors. |
Yujia Zheng; Ignavier Ng; Kun Zhang; |
| 418 | Learning to Branch with Tree MDPs Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Instead, we propose to learn branching rules from scratch with Reinforcement Learning (RL). |
Lara Scavuzzo; Feng Chen; Didier Chetelat; Maxime Gasse; Andrea Lodi; Neil Yorke-Smith; Karen Aardal; |
| 419 | Diagnosing Failures of Fairness Transfer Across Distribution Shift in Real-world Medical Settings Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we adopt a causal framing to motivate conditional independence tests as a key tool for characterizing distribution shifts. |
Jessica Schrouff; Natalie Harris; Sanmi Koyejo; Ibrahim Alabdulmohsin; Eva Schnider; Krista Opsahl-Ong; Alexander Brown; Subhrajit Roy; Diana Mincu; Christina Chen; Awa Dieng; Yuan Liu; Vivek Natarajan; Alan Karthikesalingam; Katherine Heller; Silvia Chiappa; Alexander D’Amour; |
| 420 | Practical Adversarial Multivalid Conformal Prediction Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We give a simple, generic conformal prediction method for sequential prediction that achieves target empirical coverage guarantees on adversarial data. |
Osbert Bastani; Varun Gupta; Christopher Jung; Georgy Noarov; Ramya Ramalingam; Aaron Roth; |
| 421 | Provably Expressive Temporal Graph Networks Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Specifically, novel constructions reveal the inadequacy of MP-TGNs and WA-TGNs, proving that neither category subsumes the other. We extend the 1-WL (Weisfeiler-Leman) test to temporal graphs, and show that the most powerful MP-TGNs should use injective updates, as in this case they become as expressive as the temporal WL. |
Amauri Souza; Diego Mesquita; Samuel Kaski; Vikas Garg; |
| 422 | Learning Generalizable Models for Vehicle Routing Problems Via Knowledge Distillation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present a generic and efficient Adaptive Multi-Distribution Knowledge Distillation (AMDKD) scheme to tackle the cross-distribution generalization issue for learning-to-solve routing problems. |
Jieyi Bi; Yining Ma; Jiahai Wang; Zhiguang Cao; Jinbiao Chen; Yuan Sun; Yeow Meng Chee; |
| 423 | Few-Shot Audio-Visual Learning of Environment Acoustics Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Towards that goal, we introduce a transformer-based method that uses self-attention to build a rich acoustic context, then predicts RIRs of arbitrary query source-receiver locations through cross-attention. |
Sagnik Majumder; Changan Chen; Ziad Al-Halah; Kristen Grauman; |
| 424 | UniCLIP: Unified Framework for Contrastive Language-Image Pre-training Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, as these works define inter-domain (image-text) contrastive loss and intra-domain (image-image) contrastive loss in individual spaces, many feasible combinations of supervision are overlooked. To overcome this issue, we propose UniCLIP, a Unified framework for Contrastive Language-Image Pre-training. |
Janghyeon Lee; Jongsuk Kim; Hyounguk Shon; Bumsoo Kim; Seung Hwan Kim; Honglak Lee; Junmo Kim; |
| 425 | On Convergence of FedProx: Local Dissimilarity Invariant Bounds, Non-smoothness and Beyond Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We contribute to derive several new and deeper theoretical insights into the FedProx algorithm under milder conditions. |
Xiaotong Yuan; Ping Li; |
| 426 | Shadow Knowledge Distillation: Bridging Offline and Online Knowledge Transfer Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: However, this fine-tuning process still costs lots of training budgets. To alleviate this dilemma, we propose SHAKE, a simple yet effective SHAdow KnowlEdge transfer framework to bridge offline and online distillation, which trades the accuracy with efficiency. |
Lujun Li; ZHE JIN; |
| 427 | Score-based Generative Modeling Secretly Minimizes The Wasserstein Distance Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Recently, Song et al. showed that the training objective of score-based generative models is equivalent to minimizing the Kullback-Leibler divergence of the generated distribution from the data distribution. In this work, we show that score-based models also minimize the Wasserstein distance between them. |
Dohyun Kwon; Ying Fan; Kangwook Lee; |
| 428 | Non-asymptotic and Accurate Learning of Nonlinear Dynamical Systems Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We consider the problem of learning a nonlinear dynamical system governed by a nonlinear state equation $h_{t+1}=\phi(h_t,u_t;\theta)+w_t$. |
Yahya Sattar; Samet Oymak; |
| 429 | Reinforced Genetic Algorithm for Structure-based Drug Design Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To achieve a more stable and efficient SBDD, we propose Reinforced Genetic Algorithm (RGA) that uses neural models to prioritize the profitable design steps and suppress random-walk behavior. |
Tianfan Fu; Wenhao Gao; Connor Coley; Jimeng Sun; |
| 430 | Wavelet Score-Based Generative Modeling Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We show that SGMs can be considerably accelerated, by factorizing the data distribution into a product of conditional probabilities of wavelet coefficients across scales. |
Florentin Guth; Simon Coste; Valentin De Bortoli; Stephane Mallat; |
| 431 | BiT: Robustly Binarized Multi-distilled Transformer Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we identify a series of improvements that enables binary transformers at a much higher accuracy than what was possible previously. |
Zechun Liu; Barlas Oguz; Aasish Pappu; Lin Xiao; Scott Yih; Meng Li; Raghuraman Krishnamoorthi; Yashar Mehdad; |
| 432 | MRI: Multi-modal 3D Human Pose Estimation Dataset Using MmWave, RGB-D, and Inertial Sensors Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: mRI is a large-scale multi-modal human pose estimation dataset focusing on rehab movements, supporting human pose estimation and human activity recognition tasks. |
Sizhe An; Yin Li; Umit Ogras; |
| 433 | HierSpeech: Bridging The Gap Between Text and Speech By Hierarchical Variational Inference Using Self-supervised Representations for Speech Synthesis Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper presents HierSpeech, a high-quality end-to-end text-to-speech (TTS) system based on a hierarchical conditional variational autoencoder (VAE) utilizing self-supervised speech representations. |
Sang-Hoon Lee; Seung-Bin Kim; Ji-Hyun Lee; Eunwoo Song; Min-Jae Hwang; Seong-Whan Lee; |
| 434 | Nearly Optimal Algorithms for Linear Contextual Bandits with Adversarial Corruptions Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: The best-known algorithms in this setting are limited in that they either are computationally inefficient or require a strong assumption on the corruption, or their regret is at least $C$ times worse than the regret without corruption. In this paper, to overcome these limitations, we propose a new algorithm based on the principle of optimism in the face of uncertainty. |
Jiafan He; Dongruo Zhou; Tong Zhang; Quanquan Gu; |
| 435 | Towards Efficient Post-training Quantization of Pre-trained Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Therefore, they suffer from slow training, large memory overhead, and data accessibility issues. In this paper, we study post-training quantization~(PTQ) of PLMs, and propose module-wise quantization error minimization~(MREM), an efficient solution to mitigate these issues. |
Haoli Bai; Lu Hou; Lifeng Shang; Xin Jiang; Irwin King; Michael R Lyu; |
| 436 | Learning Consistency-Aware Unsigned Distance Functions Progressively from Raw Point Clouds Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we propose a novel method to learn consistency-aware unsigned distance functions directly from raw point clouds. |
Junsheng Zhou; Baorui Ma; Yu-Shen Liu; Yi Fang; Zhizhong Han; |
| 437 | On Uncertainty, Tempering, and Data Augmentation in Bayesian Classification Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Our work shows that explicitly accounting for aleatoric uncertainty significantly improves the performance of Bayesian neural networks. |
Sanyam Kapoor; Wesley Maddox; Pavel Izmailov; Andrew Wilson; |
| 438 | On Privacy and Personalization in Cross-Silo Federated Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we instead consider the more realistic notion of \textit{silo-specific item-level} privacy, where silos set their own privacy targets for local examples. |
Ken Liu; Shengyuan Hu; Steven Wu; Virginia Smith; |
| 439 | RankFeat: Rank-1 Feature Removal for Out-of-distribution Detection Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we observe that the singular value distributions of the in-distribution (ID) and OOD features are quite different: the OOD feature matrix tends to have a larger dominant singular value than the ID feature, and the class predictions of OOD samples are largely determined by it. |
Yue Song; Nicu Sebe; Wei Wang; |
| 440 | Optimal Rates for Regularized Conditional Mean Embedding Learning Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We address the consistency of a kernel ridge regression estimate of the conditional mean embedding (CME), which is an embedding of the conditional distribution of $Y$ given $X$ into a target reproducing kernel Hilbert space $\mathcal{H}_Y$. |
Zhu Li; Dimitri Meunier; Arthur Gretton; |
| 441 | PAC-Bayes Compression Bounds So Tight That They Can Explain Generalization Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this paper, we develop a compression approach based on quantizing neural network parameters in a linear subspace, profoundly improving on previous results to provide state-of-the-art generalization bounds on a variety of tasks, including transfer learning. |
Sanae Lotfi; Sanyam Kapoor; Marc Finzi; Andres Potapczynski; Micah Goldblum; Andrew Wilson; |
| 442 | Your Transformer May Not Be As Powerful As You Expect Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we mathematically analyze the power of RPE-based Transformers regarding whether the model is capable of approximating any continuous sequence-to-sequence functions. |
Shengjie Luo; Shanda Li; Shuxin Zheng; Tie-Yan Liu; Liwei Wang; Di He; |
| 443 | Unsupervised Cross-Task Generalization Via Retrieval Augmentation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Humans can perform unseen tasks by recalling relevant skills that are acquired previously and then generalizing them to the target tasks, even if there is no supervision at all. In this paper, we aim to improve such cross-task generalization ability of massive multi-task language models such as T0 (Sanh et al., 2021) in an unsupervised setting. |
Bill Yuchen Lin; Kangmin Tan; Chris Miller; Beiwen Tian; Xiang Ren; |
| 444 | SPD Domain-specific Batch Normalization to Crack Interpretable Unsupervised Domain Adaptation in EEG Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose and evaluate (using EEG) an unsupervised domain adaptation framework around SPD domain-specific momentum batch normalization that enables end-to-end learning of tangent space mapping models. |
Reinmar Kobler; Jun-ichiro Hirayama; Qibin Zhao; Motoaki Kawanabe; |
| 445 | Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Though work to evaluate language model harms is under way, translating foresight about which harms may arise into rigorous benchmarks is not straightforward. To facilitate this translation, we outline six ways of characterizing harmful text which merit explicit consideration when designing new benchmarks. |
Maribeth Rauh; John Mellor; Jonathan Uesato; Po-Sen Huang; Johannes Welbl; Laura Weidinger; Sumanth Dathathri; Amelia Glaese; Geoffrey Irving; Iason Gabriel; William Isaac; Lisa Anne Hendricks; |
| 446 | Nocturne: A Scalable Driving Benchmark for Bringing Multi-agent Learning One Step Closer to The Real World Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce a fast, data-driven simulator for studying multi-agent partially observed coordination in human driving. |
Eugene Vinitsky; Nathan Lichtlé; Xiaomeng Yang; Brandon Amos; Jakob Foerster; |
| 447 | Make Some Noise: Reliable and Efficient Single-Step Adversarial Training Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce a novel single-step attack for adversarial training that can prevent catastrophic overfitting while obtaining a 3x speed-up. |
Pau de Jorge Aranda; Adel Bibi; Riccardo Volpi; Amartya Sanyal; Philip Torr; Gregory Rogez; Puneet Dokania; |
| 448 | VRL3: A Data-Driven Framework for Visual Deep Reinforcement Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We combine pretraining, offline RL and online RL into a 3-stage framework that makes Adroit robotic hand learning up to 24x more sample efficient than previous SOTA. |
Che Wang; Xufang Luo; Keith Ross; Dongsheng Li; |
| 449 | Learning Equivariant Segmentation with Instance-Unique Querying Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we devise a new training framework that boosts query-based models through discriminative query embedding learning. |
Wenguan Wang; James Liang; Dongfang Liu; |
| 450 | HUMUS-Net: Hybrid Unrolled Multi-scale Network Architecture for Accelerated MRI Reconstruction Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a hybrid architecture combining the efficiency of convolutions with the power of Transformers tailored for MRI reconstruction. |
Zalan Fabian; Berk Tinaz; Mahdi Soltanolkotabi; |
| 451 | Touch and Go: Learning from Human-Collected Vision and Touch Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We introduce “Touch and Go”, a human-collected dataset containing paired visual and tactile data from real-world scenes. |
Fengyu Yang; Chenyang Ma; Jiacheng Zhang; Jing Zhu; Wenzhen Yuan; Andrew Owens; |
| 452 | EgoTaskQA: Understanding Human Tasks in Egocentric Videos Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We present the EgoTaskQA benchmark that targets at action dependencies, post-effects, agents’ intents and goals, as well as multi-agent belief modeling in egocentric goal-oriented videos. |
Baoxiong Jia; Ting Lei; Song-Chun Zhu; Siyuan Huang; |
| 453 | On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with No Catastrophic Forgetting Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We describe and exploit connections between two distinct paradigms for expressing preferences over outputs of language models: reward maximization and distribution matching. |
Tomasz Korbak; Hady Elsahar; Germán Kruszewski; Marc Dymetman; |
| 454 | Learning Debiased Classifier with Biased Committee Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper proposes a new method for training debiased classifier, learning debiased classifier with biased committee (LWBC). |
Nayeong Kim; SEHYUN HWANG; Sungsoo Ahn; Jaesik Park; Suha Kwak; |
| 455 | Boosting Out-of-distribution Detection with Typical Features Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We delve into the obstacle factors in OOD detection from the perspective of typicality and propose to boost the OOD detection with typical features. |
Yao Zhu; YueFeng Chen; Chuanlong Xie; Xiaodan Li; Rong Zhang; Hui Xue’; Xiang Tian; bolun zheng; Yaowu Chen; |
| 456 | Polyhistor: Parameter-Efficient Multi-Task Adaptation for Dense Vision Tasks Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, we provide an extensive single- and multi-task parameter-efficient benchmark and examine existing parameter-efficient fine-tuning NLP methods for vision tasks. |
Yen-Cheng Liu; CHIH-YAO MA; Junjiao Tian; Zijian He; Zsolt Kira; |
| 457 | When to Trust Your Simulator: Dynamics-Aware Hybrid Offline-and-Online Reinforcement Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This brings up a new question: is it possible to combine learning from limited real data in offline RL and unrestricted exploration through imperfect simulators in online RL to address the drawbacks of both approaches? In this study, we propose the Dynamics-Aware Hybrid Offline-and-Online Reinforcement Learning (H2O) framework to provide an affirmative answer to this question. |
Haoyi Niu; shubham sharma; Yiwen Qiu; Ming Li; Guyue Zhou; Jianming HU; Xianyuan Zhan; |
| 458 | Concrete Score Matching: Generalized Score Matching for Discrete Data Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: To this end, we propose an analogous score function called the “Concrete score”, a generalization of the (Stein) score for discrete settings. |
Chenlin Meng; Kristy Choi; Jiaming Song; Stefano Ermon; |
| 459 | Towards Efficient 3D Object Detection with Knowledge Distillation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To this end, we explore the potential of knowledge distillation (KD) for developing efficient 3D object detectors, focusing on popular pillar- and voxel-based detectors. |
Jihan Yang; Shaoshuai Shi; Runyu Ding; Zhe Wang; Xiaojuan Qi; |
| 460 | Bridging The Gap Between Vision Transformers and Convolutional Neural Networks on Small Datasets Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To this end, we propose Dynamic Hybrid Vision Transformer (DHVT) as the solution to enhance the two inductive biases. |
Zhiying Lu; Hongtao Xie; Chuanbin Liu; Yongdong Zhang; |
| 461 | Is A Modular Architecture Enough? Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We highlight the benefits of modularity and sparsity and reveal insights on the challenges faced while optimizing modular systems. In doing so, we propose evaluation metrics that highlight the benefits of modularity, the regimes in which these benefits are substantial, as well as the sub-optimality of current end-to-end learned modular systems as opposed to their claimed potential. |
Sarthak Mittal; Yoshua Bengio; Guillaume Lajoie; |
| 462 | Error Correction Code Transformer Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a novel SOTA Neural error correction decoder based on Transformers. |
Yoni Choukroun; Lior Wolf; |
| 463 | Training Uncertainty-Aware Classifiers with Conformalized Deep Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper develops a novel loss function and learning algorithm for training uncertainty-aware deep neural classifiers that can lead to smaller conformal prediction sets with more reliable coverage compared to standard state-of-the-art techniques. |
Bat-Sheva Einbinder; Yaniv Romano; Matteo Sesia; Yanfei Zhou; |
| 464 | Towards A Standardised Performance Evaluation Protocol for Cooperative MARL Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We conduct a detailed meta-analysis of prior Cooperative MARL work, take inspiration from recent trends in RL and our data-driven insights to propose a standardised performance evaluation protocol for Cooperative MARL. |
Rihab Gorsane; Oumayma Mahjoub; Ruan John de Kock; Roland Dubb; Siddarth Singh; Arnu Pretorius; |
| 465 | Neural Lyapunov Control of Unknown Nonlinear Systems with Stability Guarantees Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a learning framework to simultaneously learn and stabilize an unknown nonlinear system with provable guarantees. |
Ruikun Zhou; Thanin Quartz; Hans De Sterck; Jun Liu; |
| 466 | Self-supervised Learning of Brain Dynamics from Broad Neuroimaging Data Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We devise and evaluate novel self-supervised learning techniques for neuroimaging data inspired by prominent learning frameworks in natural language processing, using one of the broadest neuroimaging datasets used for pre-training to date. |
Armin Thomas; Christopher Ré; Russell Poldrack; |
| 467 | Graph Neural Networks with Adaptive Readouts Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose differentiable and adaptive readouts for graph neural networks which replace standard operators such as sum or max, then discuss the benefits and trade-offs with an extensive empirical analysis. |
David Buterez; Jon Paul Janet; Steven J Kiddle; Dino Oglic; Pietro Liò; |
| 468 | Scalable and Efficient Training of Large Convolutional Neural Networks with Differential Privacy Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Large convolutional neural networks (CNN) can be difficult to train in the differentially private (DP) regime, since the optimization algorithms require a computationally expensive operation, known as the per-sample gradient clipping. We propose an efficient and scalable implementation of this clipping on convolutional layers, termed as the mixed ghost clipping, that significantly eases the private training in terms of both time and space complexities, without affecting the accuracy. |
Zhiqi Bu; Jialin Mao; Shiyun Xu; |
| 469 | Truncated Proposals for Scalable and Hassle-free Simulation-based Inference Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We introduce an efficient and testable simulation-based inference method that can scale to complex models with many parameters. |
Michael Deistler; Pedro Goncalves; Jakob H Macke; |
| 470 | Masked Autoencoding for Scalable and Generalizable Decision Making Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: To this end, this paper presents masked decision prediction (MaskDP), a simple and scalable self-supervised pretraining method for reinforcement learning (RL), and behavioral cloning (BC). |
Fangchen Liu; Hao Liu; Aditya Grover; Pieter Abbeel; |
| 471 | Neural Conservation Laws: A Divergence-Free Perspective Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We hence propose an approach to building divergence-free neural networks through the concept of differential forms, and with the aid of automatic differentiation, realize two practical constructions with differing trade offs. |
Jack Richter-Powell; Yaron Lipman; Ricky T. Q. Chen; |
| 472 | Better Uncertainty Calibration Via Proper Scores for Classification and Beyond Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we introduce the framework of \textit{proper calibration errors}, which relates every calibration error to a proper score and provides a respective upper bound with optimal estimation properties. |
Sebastian Gruber; Florian Buettner; |
| 473 | Multi-modal Grouping Network for Weakly-Supervised Audio-Visual Video Parsing Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: To this end, in this paper, we propose a novel Multi-modal Grouping Network, namely MGN, for explicitly semantic-aware grouping. |
Shentong Mo; Yapeng Tian; |
| 474 | Neural Basis Models for Interpretability Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose a novel subfamily of GAMs that utilizes basis decomposition of shape functions, called Neural Basis Models (NBMs). NBMs exploit the feature correlations and allow GAMs to scale by order of magnitude while preserving the interpretability. |
Filip Radenovic; Abhimanyu Dubey; Dhruv Mahajan; |
| 475 | Divert More Attention to Vision-Language Tracking Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We explore a different path to achieve SOTA tracking via vision-language multimodal learning instead of complex Transformer. |
Mingzhe Guo; Zhipeng Zhang; Heng Fan; Liping Jing; |
| 476 | Adv-Attribute: Inconspicuous and Transferable Adversarial Attack on Face Recognition Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this work, instead of performing perturbations on the low-level pixels, we propose to generate attacks through perturbing on the high-level semantics to improve attack transferability. |
Shuai Jia; Bangjie Yin; Taiping Yao; Shouhong Ding; Chunhua Shen; Xiaokang Yang; Chao Ma; |
| 477 | LASSIE: Learning Articulated Shapes from Sparse Image Ensemble Via 3D Part Discovery Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We learn to reconstruct high-quality articulated shapes from sparse image collections by discovering 3D neural parts without any shape template or keypoint annotations. |
Chun-Han Yao; Wei-Chih Hung; Yuanzhen Li; Michael Rubinstein; Ming-Hsuan Yang; Varun Jampani; |
| 478 | Augmentations in Hypergraph Contrastive Learning: Fabricated and Generative Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: This paper targets at improving the generalizability of hypergraph neural networks in the low-label regime, through applying the contrastive learning approach from images/graphs. |
Tianxin Wei; Yuning You; Tianlong Chen; Yang Shen; Jingrui He; Zhangyang Wang; |
| 479 | Gradient Descent: The Ultimate Optimizer Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We show how to \emph{automatically} compute hypergradients with a simple and elegant modification to backpropagation. |
Kartik Chandra; Audrey Xie; Jonathan Ragan-Kelley; ERIK MEIJER; |
| 480 | Efficient Adversarial Training Without Attacking: Worst-Case-Aware Robust Reinforcement Learning Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: In this work, we propose a strong and efficient robust training framework for RL, named Worst-case-aware Robust RL (WocaR-RL), that directly estimates and optimizes the worst-case reward of a policy under bounded $\ell_p$ attacks without requiring extra samples for learning an attacker. |
Yongyuan Liang; Yanchao Sun; Ruijie Zheng; Furong Huang; |
| 481 | Pre-Train Your Loss: Easy Bayesian Transfer Learning with Informative Priors Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Instead, we show that we can learn highly informative posteriors from the source task, through supervised or self-supervised approaches, which then serve as the basis for priors that modify the whole loss surface on the downstream task. |
Ravid Shwartz-Ziv; Micah Goldblum; Hossein Souri; Sanyam Kapoor; Chen Zhu; Yann LeCun; Andrew Wilson; |
| 482 | On The Difficulty of Learning Chaotic Dynamics with RNNs Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: It is particularly problematic in scientific applications where one aims to reconstruct the underlying dynamical system. Here we offer a comprehensive theoretical treatment of this problem by relating the loss gradients during RNN training to the Lyapunov spectrum of RNN-generated orbits. |
Jonas Mikhaeil; Zahra Monfared; Daniel Durstewitz; |
| 483 | Decentralized Gossip-Based Stochastic Bilevel Optimization Over Communication Networks Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: This paper studies the problem of distributed bilevel optimization over a network where agents can only communicate with neighbors, including examples from multi-task, multi-agent learning and federated learning.In this paper, we propose a gossip-based distributed bilevel learning algorithm that allows networked agents to solve both the inner and outer optimization problems in a single timescale and share information via network propagation. |
Shuoguang Yang; Xuezhou Zhang; Mengdi Wang; |
| 484 | Expansion and Shrinkage of Localization for Weakly-Supervised Semantic Segmentation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: The original CAM method usually produces incomplete and inaccurate localization maps. To tackle with this issue, this paper proposes an Expansion and Shrinkage scheme based on the offset learning in the deformable convolution, to sequentially improve the \textbf{recall} and \textbf{precision} of the located object in the two respective stages. |
JINLONG LI; Zequn Jie; Xu Wang; Xiaolin Wei; Lin Ma; |
| 485 | Segmenting Moving Objects Via An Object-Centric Layered Representation Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: The objective of this paper is a model that is able to discover, track and segment multiple moving objects in a video. |
Junyu Xie; Weidi Xie; Andrew Zisserman; |
| 486 | Learning Physical Dynamics with Subequivariant Graph Neural Networks Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose Subequivariant GNN for learning physical dynamics with desirable symmetry and object information. |
Jiaqi Han; Wenbing Huang; Hengbo Ma; Jiachen Li; Josh Tenenbaum; Chuang Gan; |
| 487 | Unsupervised Representation Learning from Pre-trained Diffusion Probabilistic Models Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Inspired by the classifier-guided sampling method, we employ an encoder to learn meaningful representations from images and a gradient estimator to directly model the mean shift according to the learned representations to fill the posterior mean gap for image reconstruction. |
Zijian Zhang; Zhou Zhao; Zhijie Lin; |
| 488 | Active Learning Through A Covering Lens Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We formulate deep active learning as a probability coverage problem, and propose an active learning algorithm that improves the state-of-the-art in low budgets. |
Ofer Yehuda; Avihu Dekel; Guy Hacohen; Daphna Weinshall; |
| 489 | On The Tradeoff Between Robustness and Fairness Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: Theoretically, we analyze the class-wise performance of adversarially trained linear models with mixture Gaussian distribution. |
Xinsong Ma; Zekai Wang; Weiwei Liu; |
| 490 | BigBio: A Framework for Data-Centric Biomedical Natural Language Processing Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: BigBio is a community library of 126+ biomedical NLP datasets, covering 13 tasks and 10 languages. |
Jason Fries; Leon Weber; Natasha Seelam; Gabriel Altay; Debajyoti Datta; Samuele Garda; Sunny Kang; Rosaline Su; Wojciech Kusa; Samuel Cahyawijaya; Fabio Barth; Simon Ott; Matthias Samwald; Stephen Bach; Stella Biderman; Mario Sänger; Bo Wang; Alison Callahan; Daniel León Periñán; Théo Gigant; Patrick Haller; Jenny Chim; Jose Posada; John Giorgi; Karthik Rangasai Sivaraman; Marc Pàmies; Marianna Nezhurina; Robert Martin; Michael Cullan; Moritz Freidank; Nathan Dahlberg; Shubhanshu Mishra; Shamik Bose; Nicholas Broad; Yanis Labrak; Shlok Deshmukh; Sid Kiblawi; Ayush Singh; Minh Chien Vu; Trishala Neeraj; Jonas Golde; Albert Villanova del Moral; Benjamin Beilharz; |
| 491 | SafeBench: A Benchmarking Platform for Safety Evaluation of Autonomous Vehicles Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We propose the first unified platform SafeBench to effectively and efficiently evaluate autonomous driving algorithms against different types of safety-critical testing scenarios. |
Chejian Xu; Wenhao Ding; Weijie Lyu; ZUXIN LIU; Shuai Wang; Yihan He; Hanjiang Hu; DING ZHAO; Bo Li; |
| 492 | Training with More Confidence: Mitigating Injected and Natural Backdoors During Training Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: By further analyzing the training process and model architectures, we found that piece-wise linear functions cause this hyperplane surface. In this paper, we design a novel training method that forces the training to avoid generating such hyperplanes and thus remove the injected backdoors. |
Zhenting Wang; Hailun Ding; Juan Zhai; Shiqing Ma; |
| 493 | Differentially Private Learning Needs Hidden State (Or Much Faster Convergence) Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we significantly improve privacy analysis under the hidden state assumption. |
Jiayuan Ye; Reza Shokri; |
| 494 | Rethinking Image Restoration for Object Detection Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Specifically, we present an ADAM-like adversarial attack to generate pseudo ground truth for restoration training. |
Shangquan Sun; Wenqi Ren; Tao Wang; Xiaochun Cao; |
| 495 | Computationally Efficient Horizon-Free Reinforcement Learning for Linear Mixture MDPs Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: In this paper, we propose the first computationally efficient horizon-free algorithm for linear mixture MDPs, which achieves the optimal $\tilde O(d\sqrt{K} +d^2)$ regret up to logarithmic factors. |
Dongruo Zhou; Quanquan Gu; |
| 496 | Gradient-Free Methods for Deterministic and Stochastic Nonsmooth Nonconvex Optimization Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: The contributions of this paper are two-fold. First, we establish the relationship between the celebrated Goldstein subdifferential~\citep{Goldstein-1977-Optimization} and uniform smoothing, thereby providing the basis and intuition for the design of gradient-free methods that guarantee the finite-time convergence to a set of Goldstein stationary points. Second, we propose the gradient-free method (GFM) and stochastic GFM for solving a class of nonsmooth nonconvex optimization problems and prove that both of them can return a $(\delta,\epsilon)$-Goldstein stationary point of a Lipschitz function $f$ at an expected convergence rate at $O(d^{3/2}\delta^{-1}\epsilon^{-4})$ where $d$ is the problem dimension. |
Tianyi Lin; Zeyu Zheng; Michael Jordan; |
| 497 | Rethinking The Reverse-engineering of Trojan Triggers Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We observe that both input-space and feature-space Trojans are associated with feature space hyperplanes.Based on this observation, we design a novel reverse-engineering method that exploits the feature space constrain to reverse-engineer Trojan triggers. |
Zhenting Wang; Kai Mei; Hailun Ding; Juan Zhai; Shiqing Ma; |
| 498 | Decentralized Local Stochastic Extra-Gradient for Variational Inequalities Related Papers Related Patents Related Grants Related Venues Related Experts View Highlight: We consider distributed stochastic variational inequalities (VIs) on unbounded domains with the problem data that is heterogeneous (non-IID) and distributed across many devices. |
Aleksandr Beznosikov; Pavel Dvurechenskii; Anastasiia Koloskova; Valentin Samokhin; Sebastian Stich; Alexander Gasnikov; |
| 499 | Causal Discovery in Heterogeneous Environments Under The Sparse Mechanism Shift Hypothesis Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: We propose the Mechanism Shift Score (MSS), a score-based approach amenable to various empirical estimators, which provably identifies the entire causal structure with high probability if the sparse mechanism shifts hypothesis holds. |
Ronan Perry; Julius von Kügelgen; Bernhard Schölkopf; |
| 500 | Multi-LexSum: Real-world Summaries of Civil Rights Lawsuits at Multiple Granularities Related Papers Related Patents Related Grants Related Venues Related Experts Related Code View Highlight: Multi-LexSum is a multi-doc summarization dataset for civil rights litigations lawsuits with summaries of three granularities. |
Zejiang Shen; Kyle Lo; Lauren Yu; Nathan Dahlberg; Margo Schlanger; Doug Downey; |
This table only includes 500 papers selected by our daily digest algorithm. To continue with the full list (~2,900 papers), please visit Paper Digest: NeurIPS-2022 (Full List).