This is a bugfix release, addressing two issues:

• Ability to save a model when a file with the same name already exists.
• Issue with loading legacy config files for the Sequential model.

See here for the changelog since 2.2.2.

This version fixes a few bugs released in 0.21.1.

This release marks the convergence of development into a stable, reference release of the framework and a shift into maintenance mode. Let's review the progress culminating in our 1.0:

• research: nearly 4,000 citations, usage by award papers at CVPR/ECCV/ICCV, and tutorials at ECCV'14 and CVPR'15
• industry: adopted by Facebook, NVIDIA, Intel, Sony, Yahoo! Japan, Samsung, Adobe, A9, Siemens, Pinterest, the Embedded Vision Alliance, and more
• community: 250+ contributors, 15k+ subscribers on github, and 7k+ members of the mailing list
• development: 10k+ forks, >1 contribution/day on average, and dedicated branches for OpenCL and Windows
• downloads: 10k+ downloads and updates a month, ~50k unique visitors to the home page every two weeks, and >100k unique downloads of the reference models
• winner of the ACM MM open source award 2014 and presented as a talk at ICML MLOSS 2015

Thanks for all of your efforts leading us to Caffe 1.0! Your part in development, community, feedback, and framework usage brought us here. As part of 1.0 we will be welcoming collaborators old and new to join as members of the Caffe core.

Stay tuned for the next steps in DIY deep learning with Caffe. As development is never truly done, there's always 1.1!

Now that 1.0 is done, the next generation of the framework—Caffe2—is ready to keep up the progress on DIY deep learning in research and industry. While Caffe 1.0 development will continue with 1.1, Caffe2 is the new framework line for future development led by Yangqing Jia. Although Caffe2 is a departure from the development line of Caffe 1.0, we are planning a migration path for models just as we have future-proofed Caffe models in the past.

Happy brewing, The Caffe Crew

:coffee:

New Features

Automatic Mixed Precision(experimental)

Training Deep Learning networks is a very computationally intensive task. Novel model architectures tend to have increasing number of layers and parameters, which slows down training. Fortunately, new generations of training hardware as well as software optimizations, make it a feasible task. However, where most of the (both hardware and software) optimization opportunities exists is in exploiting lower precision (like FP16) to, for example, utilize Tensor Cores available on new Volta and Turing GPUs. While training in FP16 showed great success in image classification tasks, other more complicated neural networks typically stayed in FP32 due to difficulties in applying the FP16 training guidelines. That is where AMP (Automatic Mixed Precision) comes into play. It automatically applies the guidelines of FP16 training, using FP16 precision where it provides the most benefit, while conservatively keeping in full FP32 precision operations unsafe to do in FP16. To learn more about AMP, checkout this tutorial.

MKL-DNN Reduced precision inference and RNN API support

Two advanced features, fused computation and reduced-precision kernels, are introduced by MKL-DNN in the recent version. These features can significantly speed up the inference performance on CPU for a broad range of deep learning topologies. MXNet MKL-DNN backend provides optimized implementations for various operators covering a broad range of applications including image classification, object detection, natural language processing. Refer to the MKL-DNN operator documentation for more information.

Dynamic Shape(experimental)

MXNet now supports Dynamic Shape in both imperative and symbolic mode. MXNet used to require that operators statically infer the output shapes from the input shapes. However, there exist some operators that don't meet this requirement. Examples are:

• while_loop: its output size depends on the number of iterations in the loop.
• boolean indexing: its output size depends on the value of the input data.
• many operators can be extended to take a shape symbol as input and the shape symbol can determine the output shape of these operators (with this extension, the symbol interface of MXNet can fully support shape). To support dynamic shape and such operators, we have modified MXNet backend including graph binding, the MXNet executor and the operator interface. Now MXNet supports operators with dynamic shape such as contrib.while_loop, contrib.cond, and mxnet.ndarray.contrib.boolean_mask Note: Currently dynamic shape does not work with Gluon defferred initialization.

Large Tensor Support

Current MXNet only supports maximal tensors size around 4 billon (2^32). This is because uint32_t is used as default data type for tensor size as well as indexing variables. This limitation has created many problems when larger tensors are used in the model. A naive solution to this problem is to replace all uint32_t in the MXNet backend source code by int64_t. This solution is not viable, however, because many data structures use uint32_t as data type for its members. Unnecessarily replacing these variables to int64_t will increase the memory consumption causing another limitation. Second, MXNet has many submodule dependencies. Updating the variable types in MXNet repository is not enough. We also need to make sure different libraries, such as MKLDNN, MShadow etc. supports the int64_t integer data type. Third, many front end APIs assumes unsigned 32-bit integer interface. Only updating the interface in C/C++ will cause all the language bindings to fail. Therefore, we need a systematic approach to enhance MXNet to support large tensors. Now you can enable large tensor support by changing the following build flag to 1: USE_INT64_TENSOR_SIZE = 1. Note this is set to 0 by default. For more details please refer to the design document.

Dependency Update

MXNet now added support for CUDA 10, CUDA 10.1, cudnn7.5, NCCL 2.4.2, numpy 1.16.0 to benefit from the latest features and performance improvements. These updates are available through PyPI packages and build from source, refer to installation guid for more details.

Gluon Fit API(experimental)

Training a model in Gluon requires users to write the training loop, this is useful because of its imperative nature, however repeating the same code across multiple models can become tedious and repetitive with boilerplate code. The training loop can also be overwhelming to some users new to deep learning. We have introduced a Estimator and Fit API to help facilitate training loop. Note: this feature is still experimental, for more details, refer to design document.

New Operators

• split_v2 (#13687)
• Gradient multiplier (contrib) operator (#13632)
• Image normalize operator - GPU support, 3D/4D inputs (#13802)
• Image ToTensor operator - GPU support, 3D/4D inputs (#13837)
• Add Gluon Transformer Crop (#14259)
• GELU (#14449)
• AdamW operator (Fixing Weight Decay Regularization in Adam) (#13728)
• MXNET-1382 Add the index_array operator (#14638)
• add an operator for computing the likelihood of a Hawkes self-exciting process (#14683)
• Add numpy linspace (#14927)

Feature Improvements

Operators

• make ROIAlign support position-sensitive pooling (#13088)
• Add erfinv operator for calculating inverse error function (#13811)
• Added optional parameters to BilinearResize2D to do relative scaling (#13985)
• MXNET-1295 Adding integer index support to Sequence* family of operators. (#13880)
• Export resize and support batch size (#14014)
• CUDNN dropout (#13896)
• Relaxing type requirements for slice_like op (#14097)
• Relaxing type requirements for reshape_like op (#14325)
• Parallelize CPU version and add GPU version of boolean_mask op (#14090)
• Add NHWC layout support to Pooling (cpu, gpu cuda, gpu cuDNN) (#13749)
• Multi-precision AdamW update op (#14171)
• op add back support for scalar type rescale_grad argument for adamw_update/mp_adamw_update (#14221)
• move choose_element_0index to operator (#14273)
• Optimize NMS (#14290)
• Optimize NMS part 2 (#14352)
• add backgroud class in box_nms (#14058)
• Use cudnn for dropout by default (#14278)
• In-place updates for Nadam, Adadelta, Adamax and SGLD (#13960)
• Aggregate SGD (#13346)
• Add proper exception message for negative shape in array creation routines (#14362)
• Support multi-threading for Custom Operator (#14363)
• moveaxis operator now accepts negative indices and sequence of ints as well. (#14321)
• Support SyncBatchNorm5D (#14542)
• Add nd.power and sym.pow (#14606)
• Change RNN OP to stateful (#14476)
• Add imresize and copyMakeBorder to mx.image (#13357)
• add ctx for rand_ndarray and rand_sparse_ndarray (#14966)
• Add cpu implementation for Deformable PSROIPooling (#14886)
• Add warning for fp16 inputs with MXNET_SAFE_ACCUMULATION=0 (#15046)
• Safe LayerNorm (#15002)
• use MXNET_SAFE_ACCUMULATION for softmax accumulator (#15037)
• LayerNorm acceleration on GPU (#14935)
• Add matrix inversion operator in linalg (#14963)
• implementation for equivalence of tf.moments (#14842)
• Use env var to enforce safe accumulation in ReduceAxesCompute (#14830)
• MXNet-1211 Factor and "Like" modes in BilinearResize2D operator (#13226)
• added extraction/generation of diagonal and triangonal matrices to linalg (#14501)
• Mxnet-1397 Support symbolic api for requantize and dequantize (#14749)
• MXNET-978 Support higher order gradient for log. (#14992)
• Add cpu implementation for Deformable Convolution (#14879)

MKLDNN

• Feature/mkldnn static (#13628)
• Feature/mkldnn static 2 (#13503)
• support mkl log when dtype is fp32 or fp64 (#13150)
• Add reshape op supported by MKL-DNN (#12980)
• Move the debug output message into MXNET_MKLDNN_DEBUG (#13662)
• Integrate MKLDNN Conv1d and support 3d layout (#13530)
• Making MKL-DNN default on MXNet master (#13681)
• Add mkldnn OP for slice (#13730)
• mkldnn s8 conv API change for master (#13903)
• MKLDNN Enable signed int8 support for convolution. (#13697)
• add mkldnn softmax_output (#13699)
• MKLDNN based Quantized FullyConnected Operator and its fusion (#14128)
• Fix entropy for uint8 (#14150)
• Update MKL-DNN to v0.18 release (was: fix the Dense layer issue) (#13668)
• MKL-DNN Enable s8 support for inner product and 3d input with flatten=false (#14466)
• Optimize transpose operator with MKL-DNN (#14545)
• MKLDNN Remove repeat parts in MKLDNN.md (#14995)
• MKLDNN Enable more convolution + activation fusion (#14819)
• Update MKL-DNN submodule to v0.19 (#14783)
• Add mkldnn_version.h to pip package (#14899)
• MKLDNN add quantized sum (#14614)
• MKLDNNRefactor requantize to speed up execution (#14608)
• MKLDNNAdd quantized relu (#14604)
• Add MKLDNN headers to pip package (#14339)
• add symbolic link to mkldnn header files in include (#14300)
• disable default MKLDNN for cross compilation (#13893)
• Update MKLDNN_README.md (#13653)
• Quantization Support zero-size tensor input for quantization flow (#15031)
• Support 3D input for MKL-DNN softmax operator (#14818)
• Add primitive cache for MKL-DNN sum(elemwise_add operator (#14914)
• Fix reshape to add in-place back (#14903)
• int8 Add MobileNetV2_1.0 & ResNet18 Quantization (#14823)
• MKLDNNImprove quantizeV2 and dequantize latency (#14641)
• added mkldnn dependency for plugin compile target (#14274)
• Support Quantized Fully Connected by INT8 GEMM (#12922)

ONNX

• ONNX export: Instance normalization, Shape (#12920)
• ONNX export: Logical operators (#12852)
• ONNX import/export: Size (#13112)
• ONNX export: Add Flatten before Gemm (#13356)
• ONNX import/export: Add missing tests, ONNX export: LogSoftMax (#13654)
• ONNX import: Hardmax (#13717)
• MXNET-898 ONNX import/export: Sample_multinomial, ONNX export: GlobalLpPool, LpPool (#13500)
• ONNX ops: norm exported and lpnormalization imported (#13806)
• MXNET-880 ONNX export: Random uniform, Random normal, MaxRoiPool (#13676)
• ONNX export: Add Crop, Deconvolution and fix the default stride of Pooling to 1 (#12399)
• onnx export ops (#13821)
• ONNX export: broadcast_to, tile ops (#13981)
• ONNX export: Support equal length splits (#14121)

TensorRT

• MXNET-1252 Decouple NNVM to ONNX from NNVM to TenosrRT conversion (#13659)
• MXNET-703 Update to TensorRT 5, ONNX IR 3. Fix inference bugs. (#13310)
• MXNET-703 Minor refactor of TensorRT code (#13311)
• reformat trt to use subgraph API, add fp16 support (#14040)

FP16 Support

• Update mshadow to support batch_dot with fp16. (#13716)
• float32 → float16 cast consistency across implementations (#13857)
• modifying SyncBN doc for FP16 use case (#14041)
• support dot(vector, vector) for fp16 inputs on GPU (#14102)
• softmax for fp16 with fp32 accumulator (#14098)
• MXNET-1327 Allow RNN Layers to be initialized to fp16 (#14219)
• fp16 safe norm operator (#14616)
• NAG Optimizer with multi-precision support (#14568)

Deep Graph Library(DGL) support

• Add graph_compact operator. (#13436)
• Accelerate DGL csr neighbor sampling (#13588)

Horovod Integration

• Add extra header file to export for error checking (#13795)
• whitelist symbols for using MXNet error handling externally (#13812)
• Use CPUPinned context in ImageRecordIOParser2 (#13980)
• Add pin_device_id option to Gluon DataLoader (#14136)

Dynamic Shape

• MXNET-1315 Add checks for dynamic-shaped operators in CachedOp (#14018)
• MXNET-1325 Make InferShapeAttr a standalone pass (#14193)
• MXNET-1324 Add NaiveRunGraph to imperative utils (#14192)
• MXNET-1352 Allow dynamic shape in while_loop and if conditionals (#14393)

Backend Engine

• Add infer_type_partial (#14214)
• Tidy up storage allocation and deallocation (#14480)
• Add MXEnginePushAsync and MXEnginePushSync C APIs (#14615)
• Enhance subgraph API (#14113)
• Enhance PartitionGraph (#14277)
• Allow clearing gpu cache (#14252)
• Fix warning / static function in header. (#14900)
• Simplify creation of NodeEntry instances and use emplace_back (#14095)
• Add unpooled gpu memory type (#14716)
• MXNET-1398 Enable zero-copy from numpy to MXNet NDArray (#14733)
• Use DEFAULT macro in C APIs (#14767)
• Avoid uneccesary vector copies in imperative_utils.cc (#14665)
• Support populating errors back to MXNet engine in callback (#13922)
• Restore save/load ndarray to 1.4.1 (#15073)
• Enable serializing/deserializing ndarrays in np_shape semantics (#15090)
• numpy Support zero-dim and zero-size tensors in MXNet (#14661)
• Rename np_compat to np_shape (#15063)
• MXNET-1330 Bring nnvm::Tuple to mxnet::Tuple (#14270)

Large Tensor Support

• Large array support for randint (#14242)
• MXNET-1185 Support large array in several operators (part 1) (#13418)
• MXNET-1401 adding more operators to test support for Large Tensor (#14944)
• MXNET-1410Adding Large Tensor Support for tensor transpose (#15059)

Quantization

• Exclude concat layer for gpu quantization (#14060)
• Enhance gpu quantization (#14094)
• Register fake grad to subgraph and quantized operators (#14275)
• Add int8 data loader (#14123)

Profiler

• MXNET-857 Add initial NVTX profiler implementation (#12328)

CoreML

• Add more support for mxnet_to_coreml (#14222)

Front End API

Gluon

• Add pixelshuffle layers (#13571)
• MXNET-766 add dynamic_unroll RNN for HybridBlock (#11948)
• add pos_weight for SigmoidBinaryCrossEntropyLoss (#13612)
• Rewrite dataloader with process pool, improves responsiveness and reliability (#13447)
• Complimentary gluon DataLoader improvements (#13606)
• Fit-API Adress PR comments (#14885)
• Fit API update estimator (#14849)
• MXNET-1396 Update default handler logic (#14765)
• Fit API improve event handlers (#14685)
• move to gluon contrib (#14635)
• move estimator to contrib (#14633)
• MXNet-1340Update train stats (#14494)
• MXNet-1334base class for estimator and eventhandler (#14346)
• MXNET-1333 Estimator and Fit API (#14629)
• Add support for fast variable-length LSTM (#14208)
• Add the Gluon Implementation of Deformable Convolution (#14810)
• hybridize rnn and add model graph (#13244)

Python

• Python BucketingModule bind() with grad_req = 'add' (#13984)
• Refine runtime feature discovery python API and add documentation to … (#14130)
• Runtime feature detection (#13549)
• Add dtype visualization to plot_network (#14066)
• MXNET-1359 Adds a multiclass-MCC metric derived from Pearson (#14461)
• support long for mx.random.seed (#14314)
• Optimization of metric evaluation (#13471)
• MXNET-1403 Disable numpy's writability of NDArray once it is zero-copied to MXNet (#14948)
• Refactor ImageRecordIter (#14824)

Language Bindings

Scala

• MXNET-1260 Float64 DType computation support in Scala/Java (#13678)
• MXNET-1000 get Ndarray real value and form it from a NDArray (#12690)
• Now passing DType of Label downstream to Label's DataDesc object (#14038)
• Scala interpreter instructions (#14169)
• Add default parameters for Scala NDArray.arange (#13816)
• MXNET-1287 Up scala comp (#14667)
• MXNET-1385 Improved Scala Init and Macros warning messages (#14656)
• Remove all usages of makefile for scala (#14013)
• Update scala-package gitignore configuration. (#13962)
• MXNET-1177Adding Scala Demo to be run as a part of Nightly CI (#13823)
• MXNET-1287 Miscellaneous Scala warning fixes (#14658)
• Fix jar path and add missing ones for spark jobs (#14020)
• MXNET-1155 Add scala packageTest utility (#13046)
• MXNET-1195 Cleanup Scala README file (#13582)
• Add scalaclean to make clean (#14322)
• Add maven wraper to scala project. (#13702)
• Add new Maven build for Scala package (#13819)
• MXNET-1287 Feat dep (#14668)
• add Apache header on all XML (#14138)
• update the version name (#14076)
• change to compile time (#13835)
• MXNET-918 Random module (#13039)
• Avoid secondary deployment of package to local (#14647)

Java

• MXNET-1180 Java Image API (#13807)
• MXNET-1285 Draw bounding box with Scala/Java Image API (#14474)
• Add BERT QA Scala/Java example (#14592)
• MXNET-1232 fix demo and add Eclipse support (#13979)
• MXNET-1331 Removal of non-MXNET classes from JAR (#14303)
• Java install info update (#13912)
• MXNET-1226 add Docs update for MXNet Java (#14395)
• MXNET-1383 Java new use of ParamObject (#14645)
• MXNET-1302 Exclude commons-codec and commons-io from assembled JAR (#14000)

C++

• print error message for mxnet::cpp::Operator::Invoke when failed (#14318)
• build docs with CPP package (#13983)
• Update inception_inference.cpp (#14674)
• Optimize C++ API (#13496)

Clojure

• Clojure - Add Spec Validations to the Optimizer namespace (#13499)
• Clojure Add Spec Validations for the Random namespace (#13523)
• Clojure Correct the versions in the README so they correspond to the latest maven.org release ([#13507)
• Port of scala infer package to clojure (#13595)
• Clojure example for fixed label-width captcha recognition (#13769)
• Update project.clj file to use the snapshots repo to be able to pull (#13935)
• Clojure Add resource scope to clojure package (#13993)
• clojure-package improve docstrings in image.clj (#14307)
• Clojure Helper function for n-dim vector to ndarray (#14305)
• clojure: add comp-metric based on CompositeEvalMetric (#14553)
• Clojure enhance draw bounding box (#14567)
• Clojure Add methods based on NDArrayAPI/SymbolAPI (#14195)
• Clojure Clojure BERT QA example (#14691)
• clojure-package add ->nd-vec function in ndarray.clj (#14308)
• Clojure Correct the versions in the README so they correspond to the latest maven.org release (#13507)
• Update version to v1.5.0 including clojure package (#13566)
• clojure ndarray/symbol api random merged (#14800)
• upgrade codox to work with lein 2.9.0 (#14133)
• clojure fix: image test does not rely on s3 to run (#15122)

Julia

• Julia v0.7/1.0 support and drop v0.6 support (#12845)
• Julia: split ndarray.jl into several snippets (#14001)
• Julia: split symbolic-node.jl into several snippets (#14024)
• Julia: rename mx.clip to clamp for NDArray (#14027)
• Julia: add binding for runtime feature detection (#13992)

Perl:

• Two more gluon loss classes. (#14194)

R

• add NAG optimizer to r api (#14023)
• R-Package Makefile (#14068)

Performance Improvements

• Less cudaGet/SetDevice calls in Gluon execution (#13764)
• Improve bulking in Gluon (#13890)
• Increase perfomance of BulkAppend and BulkFlush (#14067)
• Performance improvement in ToTensor GPU Kernel (#14099)
• Performance improvement in Normalize GPU Kernel (#14139)
• Bulked op segments to allow Variable nodes (#14200)
• Performance improving for MKL-DNN Quantized FullyConnected (#14528)
• speedup SequenceMask on GPU (#14445)
• Dual stream cudnn Convolution backward() with MXNET_GPU_WORKER_NSTREAMS=2. (#14006)
• Speedup _contrib_index_copy (#14359)
• use mkl sparse matrix to improve performance (#14492)
• Re-enable static cached_op optimization (#14931)
• Speed up SequenceReverse (#14627)
• Improve FC perf when no_bias=False (#15033)
• Improve cached_op performance for static mode (#14785)

Example and Tutorials

• MXNET-949 Module API to Gluon API tutorial (#12542)
• Support SSD f32/int8 evaluation on COCO dataset (#14646)
• MXNET-1209 Tutorial transpose reshape (#13208)
• Clojure Add Fine Tuning Sentence Pair Classification BERT Example (#14769)
• example/ssd/evaluate/eval_metric.py (#14561)
• Add examples of running MXNet with Horovod (#14286)
• Added link to landing page for Java examples (#14481)
• Update lip reading example (#13647)
• MXNET-1121 Example to demonstrate the inference workflow using RNN (#13680)
• MXNET-1301 Remove the unnecessary WaitAll statements from inception_inference example (#13972)
• Modifying clojure CNN text classification example (#13865)
• MXNET-1210 Gluon Audio - Example (#13325)
• add examples and fix the dependency problem (#13620)
• add quantization example to readme (#14186)
• Add an inference script providing both accuracy and benchmark result for original wide_n_deep example (#13895)
• Update autoencoder example (#12933)
• #13813 examples with opencv4/origami (#13813)
• MXNET-1083 Add the example to demonstrate the inference workflow using C++ API (#13294)
• Add tutorial on how to use build from source jar (#14197)
• Gluon end to end tutorial (#13411)
• Update MXNetTutorialTemplate.ipynb (#13568)
• Simplifications and some fun stuff for the MNIST Gluon tutorial (#13094)
• Clarify dependency on OpenCV in CNN Visualization tutorial. (#13495)
• Update row_sparse tutorial (#13414)
• add clojure tutorials to index (#14814)
• Update lstm_crf.py (#14865)

Website

• Version switching user experience improvements (#13921)
• fix toctree Sphinx errors (#13489)
• fix link (#15036)
• fix website build (#14148)
• Fixed mailing list addresses (#13766)
• website publish updates (#14015)
• use relative links; update links (#13741)
• update social media section (#13705)
• MXNET Updated http://data.dmlc.ml/ links to http://data.mxnet.io/ (#15065)

Documentation

• MXNET-1402 MXNet docs change for 1.4.1 release (#14949)
• Add API documentation for upsampling operator with examples (#14919)
• Make docblocks for Gluon BatchNorm and SyncBatchNorm consistent with the code (#14840)
• DOC Update ubuntu install instructions from source (#14534)
• Clojure Better api docstrings by replacing newlines (#14752)
• Fix documentation for bilinear upsampling and add unit test (#14035)
• Updated docs for R-package installation (#14269)
• docstring improve docstring and indentation in module.clj (#14705)
• The folder python-howto was removed in an earlier commit. The reference to that folder was not removed. Making a PR to remove the reference to this folder to keep documents consistent (#14573)
• Updated documentation about nightly tests (#14493)
• Doc Start the tutorials for MKL-DNN backend (#14202)
• DOC fix sym.arange doc (#14237)
• fix render issue in NDArray linalg docs (#14258)
• clojure-package fix docstrings in normal.clj (#14295)
• DOC Refine documentation of runtime feature detection (#14238)
• MXNET-1178 updating scala docs (#14070)
• Fix website scala doc (#14065)
• Return value docs for nd.random. and sym.random. (#13994)
• Fixing the doc for symbolic version of rand_zipfian (#13978)
• fix doc of take operator (#13947)
• beta doc fixes (#13860)
• MXNET-1255 update hybridize documentation (#13597)
• Update Adam optimizer documentation (#13754)
• local docs build feature (#13682)
• gluon docfix (#13631)
• Added javadocs and improved example instructions (#13711)
• MXNET-1164 Generate the document for cpp-package using Doxygen (#12977)
• Fix warning in waitall doc (#13618)
• Updated docs for randint operator (#13541)
• Update java setup docs for 1.4.0 (#13536)
• clarify ops faq regarding docs strings (#13492)
• MXNET-1158 JVM Memory Management Documentation (#13105)
• Fixing a 404 in the ubuntu setup doc (#13542)
• Fix READMEs for examples (#14179)
• Doc Add MKL-DNN operator list (#14891)
• Fixed some typos in AvgPooling Docs (#14324)
• doc fix (#13465)
• Change Straight Dope to Dive into Deep Learning (#14465)
• DEV update code owner (#14862)
• Add notes about debug with libstdc++ symbols (#13533)
• Mention additional language bindings and add links (#14798)
• add contributors from intel (#14455)
• what's new - add 1.4.0 release (#14435)
• added note about cuda9.2 requirement (#14140)
• Remove unnecessary "also" in README.md (#14543)
• Updated news.md with the latest mkldnn submodule version (#14298)
• add new cloud providers to install page (#14039)
• Update NOTICE (#14043)
• Update README.md (#13973)
• Update profiler doc (#13901)
• Add CODEOWNERS for Julia package (#13872)
• update code owner (#13737)
• Update git clone location to apache github (#13706)
• NEWS.md backport from v1.4.x to master (#13693)
• Update CODEOWNERS, add Pedro Larroy. (#13579)
• MXNET-1225 Always use config.mk in make install instructions (#13364)
• Docs & website sphinx errors squished 🌦 (#13488)
• add Qing's Key to master (#14180)
• add KEY for zachgk (#14965)
• corrected a spellign (#14247)
• 1.4 release (#14297)

Build and Test

• Fix scala doc build break for v1.3.1 (#13820)
• Adds additional CUDA build environments (#14909)
• Pins version of scikit-learn for python2 due to drop in support (#14928)
• upgrade the libpng to 1.6.35 (#14620)
• Updates to cudnn package installation (#14923)
• Improve order of execution of install scripts. (#14867)
• Installs qemu pip requirements from qemu requirements file (#14355)
• update raspberry pi install instructions (#14172)
• update the scala installation tutorial on intellij (#14033)
• Removes unneeded nvidia driver ppa installation (#13814)
• script for installing gpu libraries and build tools (#13646)
• Set install path for libmxnet.so dynamic lib on Mac OS (#13629)
• compatibility with opencv4 (#14313)
• Flaky test #14189 (#14190)
• Enforce determinism for backwards compatibility checker (#14463)
• Change CUB submodule to track Nvidia CUB project. (#13322)
• Updates gpu tests to use CUDNN_VERSION supplied by the environment but default to 7.0.3 if not set (#14595)
• upgrade the version to 2.0.2 (#14621)
• Dependency Update Upgrade the libtiff to 4.0.10 (#14623)
• Dependency Update Upgrade cuDNN & NCCL (#14884)
• Dependency Update Upgrade openssl to 1.1.1b (#14837)
• Dependency Update Upgrade CI to use latest cuDNN (#14950)
• GPU RNN to use TempSpace resource for workspace. (#15056)
• Add vim-nox to ci/docker/install/ubuntu_core.sh (#14632)
• Fix dockerized GPU builds in dev_menu (#14603)
• MXNET-1093 Add python3 Docker images for each MXNet release (#12791)
• increased docker shared memory (#14119)
• Fix permissions of ci/docker/install/ubuntu_publish.sh (#13840)
• Dockerfiles for Publish Testing (#13707)
• Fix test randint (#14990)
• Silence excessive mkldnn logging output on tests. (#14947)
• Fix test memory with ResourceScope (#14666)
• Sync Horovod distributed training examples with latest changes (#14748)
• use mx.context.num_gpus instead of mx.test_utils.list_gpus in MF recommender example (#14926)
• MXNET-1400 adding tests cases to verify large tensor support for depth_to_space and space_to_depth (#14797)
• rewrite test_custom_op_exc (#14878)
• Clojure Remove unneeded test files (#14813)
• Use correct stash name when running nightly tests (#14809)
• julia/ndarray: fix flaky test cases for clamp (#14776)
• Updates tolerances for test_layer_bidirectional (#14682)
• Adds context parameter to check_rnn_layer_forward calls in test_lstmp (#14529)
• reenable the test (#14483)
• temporarily disable integ tests with a dependency on origami repo (#14448)
• Bypass ThreadedEngine in test_operator_gpu.py:test_convolution_multiple_streams. (#14338)
• Updated the MLP test to accept the number of epochs. Reduced the epochs in ci_test.sh to shorten the CI build time (#14149)
• follow up on fix nightly test (#14134)
• Julia: enable integration test (#14025)
• fix test_depthwise_convoltuion for occasional CI failures (#14016)
• fix test_stn (#14063)
• Add a test for SGLD optimizer with comparisons for set noise seeds. (#13762)
• Code modification for testcases of various network models in directory example (#12498)
• Remove MXNET_STORAGE_FALLBACK_LOG_VERBOSE from test_autograd.py (#13830)
• MXNET-1263 Unit Tests for Java Predictor and Object Detector APIs (#13794)
• ONNX test code cleanup (#13553)
• #13385 Clojure - Turn examples into integration tests (#13554)
• add cpp example inception to nightly test (#13534)
• Fix flaky test test_random:test_randint_generator (#13498)
• Adding test for softmaxoutput (#13116)
• MXNET-1235 Add a test for AdaMax optimizer (#13467)
• MXNET-545 Fix broken cython build (#10951)
• Update mkldnn window build instructions in MKLDNN_README.md (#14952)
• Added USE_SIGNAL_HANDLER to other Linux builds which didn't had it (#14122)
• Static build for Python (#13916)
• Julia: add windows-cpu build (#13937)
• Static build instruction for MXNet in general (#13914)
• Jenkins nightly maven with static build script and gpu (#13767)
• Re-organize Scala maven build (#13626)
• disable error checking when building old versions (#13725)
• scripts for building libmxnet binary and wheel (#13648)
• Improve dev_menu usability, local build and virtualenv (#13529)
• Scripts for building dependency libraries of MXNet (#13282)
• MXNET-1224: improve scala maven jni build and packing. (#13493)
• fix compile error in debug mode (#13873)
• add ccache to docs build (#13832)
• Decreases test sensitivity (#15014)
• bump up atol for test_bilinear_resize_op (#15011)
• Add STL checks via -D_GLIBCXX_ASSERTIONS in debug mode (#14896)
• clean up duplicate cudnn installation (#14996)
• fix custom op fork test (#14753)
• fix pi instructions (#14746)
• Reenable TensorRT step (#14654)
• Fixes for CI downloads (#14504)
• Fixed tutorial warnings (#14472)
• Fixes static build script for cub directory rename (#14578)
• add a compiler flag to use int64 as tensor size (#14570)
• Upgrade Pylint version to 2.3.1 (#14807)
• Fixes installation nightly test by filtering out the git commands (#14144)
• fix nightly test on tutorials (#14036)
• Fix MXNet R package build (#13952)
• re-enable test after issue fixed https://github.com/apache/incubator-mxnet/issues/10973 (#14032)
• Add back R tests and fix typo around R and perl tests (#13940)
• Fix document build (#13927)
• Temporarily disables windows pipeline to unblock PRs (#14261)
• Fix USE_MKLDNN check in Makefile (#13775)
• Fix spelling in threaded_engine_test (#14709)
• Fix cmake options parsing in dev_menu (#13458)
• Add Local test stage and option to jump directly to menu item from commandline (#13809)
• Add CPU test coverage and refine cmake builds (#13338)
• ONNX test code cleanup - part 2 (#13738)
• Rearrange tests written only for update_on_kvstore = True (#13514)
• add batch norm test (#13625)
• Adadelta optimizer test (#13443)
• Skip flaky test https://github.com/apache/incubator-mxnet/issues/13446 (#13480)
• Comment out test_unix_python3_tensorrt_gpu step (#14642)
• Enable bulking test on windows (#14392)
• rewrote the concat test to avoid flaky failures (#14049)
• #13624 clojure nightly tests (#13624)
• Temporarily disable website testing (#13887)
• adding tolerance to flaky test (#13850)
• Add publish test of PyPi cu100mkl (#14637)
• CMake: Enable installation of cpp-package headers (#13339)
• Use USE_SIGNAL_HANDLER by default set to ON in CMakeLists.txt (#14599)
• Improve CMake handling of sse2 and sse3 (#14757)
• Update base CUDA image for CI to v10.0 cuDNN 7.3.1 (#14513)
• Updates build_lib.sh to copy the cub library license (#14347)
• Add license check to dev_menu, docs build with docker (#14166)
• Print reproduction command on CI failure (#14815)
• change mxnet_option behavior (#14743)
• DEP upgrade dmlc-core (#14510)
• Use ubuntu_rat container for rat check (#14678)
• Added repeats for github status updates (#14530)
• add filter to warnings (#14532)
• CI Changes for Codified Windows AMIs (#14336)
• Refactors USE_NVRTC setting to ENABLE_CUDA_RTC in pip make config files (#14250)
• pypi package description. manifest/setup.py update (#14255)
• make rat-excludes compliant with apache release policy (#14142)
• Add libhdf5-dev to ubuntu_core.sh (#14079)
• Added logging to GitHub commit status publishing (#13615)
• CI Prevent timeouts when rebuilding containers with docker. (#13818)
• MXNET-862 Basic maven jenkins pipeline (#13450)
• Scope requests so it's not needed for dev_menu (#13771)
• Add timeout/retry logic to docker cache download (#13573)
• turn on Sphinx warnings as errors (#13544)
• MXNET-1251 Basic configuration to do static-linking (#13621)
• Improve CCache handling (#13456)
• build config for maven and pip (#13556)
• Add Intel MKL blas to Jenkins (#13607)
• Add workspace cleaning after job finished (#13490)
• Add a retry to qemu_provision (#13551)
• Deprecate Jenkinsfile (#13474)
• MXNET-1408 Adding test to verify Large Tensor Support for ravel and unravel (#15048)
• move amp test and change op support to warning (#15085)
• Fixes call to build ubuntu gpu in nightly tests (#14964)
• rat check make target (#15127)
• add epsilon for tolerance level (#15098)
• Change mx.test_utils.list_gpus to mx.context.num_gpus where possible (#14946)
• bump up cudnn to 7.5.1 & nccl 2.4.2 (#14988)
• Disables TensorRT build step (#14958)
• disable flaky integration test (#14151)
• Disables large tensor size cpu test step (#14982)
• Disable Flaky Test test_poisson_generator (#14540)
• Disabled flaky test test_negative_binomial_generator (#13784)
• Disabled flaky test test_gluon_data.test_recordimage_dataset_with_data_loader_multiworker (#13527)

Bug-fixes

• Improve dev_menu virtualenv handling (#14788)
• Fallback to dense version for grad(reshape), grad(expand_dims) (#13599)
• Fix the bug of BidirectionalCell (#13575)
• set _scale in Trainer using optimizer rescale_grad (#14593)
• MXNET-1379 update reshape operator (#14600)
• Add repr for SymbolBlock (#14423)
• Cudnn conv dgrad algo filtering (#14310)
• Fix memory leak for size-zero ndarray (#14365)
• Fixes the test_sgld (#14473)
• Revert "Fix memory leak for size-zero ndarray (#14365)" (#14477)
• fix custom operation in fork (#14451)
• Fixes test_operator_gpu.test_multinomial_generator (#14475)
• support leading dimension of -1 in ravel/unravel (#14356)
• begin=end not a valid input (#14403)
• Fix NaN value comparisons in relu, max and min ops (#14262)
• fix engine crash in shutdown phase (#14382)
• fix OOM error during resource allocation (#14444)
• Fix relative difference scala (#14417)
• Correct update count with Gluon trainer and update_on_kvstore=False (#14377)
• Fix crashes on visualization (#14425)
• Reorder module import orders for dist-kvstore (#13742)
• Fixes for trainer with update_on_kvstore=False (#13721)
• Fix errors in docstrings for subgraph op; use code directive (#13463)
• Add resiliency to onnx export code (#13426)
• update github location for sampled_block.py (#13508)
• Revert "Manually track num_max_thread (#12380)" (#13501)
• Revert "Feature/mkldnn static 2 (#13503)" (#13540)
• MXNET-1110 Add header files required by horovod (#13062)
• MXAPPS-1020 Clean up some Sphinx warnings. (#13539)
• MXNET-1249 Fix Object Detector Performance with GPU (#13522)
• MXNET-769 Use MXNET_HOME in a tempdir in windows to prevent access denied due t… (#13531)
• Chi_square_check for discrete distribution fix (#13543)
• Fix use-before-assignment in convert_dot (#13511)
• fix the situation where idx didn't align with rec (#13550)
• fix link for gluon model zoo (#13583)
• Fix exception handling api doc (#13519)
• MXNET-1253 fix control_flow_op (#13555)
• fix the Float not showing correctly problem (#13617)
• fix quantize pass error when the quantization supported Op are excluded in the model (#13596)
• Fix for import mxnet taking long time if multiple process launched (#13602)
• Revert "Feature/mkldnn static (#13628)" (#13638)
• updated reference to Apache MXNet (#13645)
• Fix incorrect delete in MXExecutorReshape exception handling (#13376)
• add build fix for Scala/Java build (#13655)
• remove omp which can cause ssd accuracy variance (#13622)
• Fix Jetson compilation (#13532)
• Revert "Fix Jetson compilation" (#13665)
• Fix Jetson compilation (#13666)
• Revert "Revert "MXNET-43 Fix Jetson compilation" (#13665)" (#13672)
• fix unpicklable transform_first on windows (#13686)
• Fix NDArray ToDLPack Bug (#13698)
• Fix the quantization script to support Python2 (#13700)
• Update basic_layers.py (#13732)
• MXNET-1231 Allow not using Some in the Scala operators (#13619)
• MXNET-244 Work around likely compiler bug on nested inlines and temporary acces… (#13535)
• Use curl to download sample data instead of wget. (#13761)
• fix bipartite match memory corruption (#13727)
• remove attributs clear on TRT nodes for GetOptimizedSymbol (#13703)
• fix redirection issues; set default version to master (#13796)
• fix for params with no dims in onnx (#13413)
• Remove semicolon in libmxnet.sym file (#13822)
• remove useless code (#13777)
• Fixing a symlink issue with R install (#13708)
• fix minor indentation (#13827)
• Fix Tree Reduction on new instance type p3dn.24xlarge (#13852)
• Clojure package infer tweaks (#13864)
• Fix cpp examples build on Mac. (#13826)
• Fix launch bounds in spatial transformer (#13188)
• Update example scripts classpath. (#13849)
• fix ssd quantization script error (#13843)
• Avoid adding SegfaultLogger if process already has sig handler. (#13842)
• fix the fetching GPU problem (#13889)
• Fix SN-GAN example doc (#13877)
• update Spectral Normalization Code (#13868)
• Fixed java benchmark failing error by fixing the classpath (#13891)
• Fix the order of error term's operands (#13745)
• fix bug in nag optimizer (#13683)
• Fix BatchNorm converter for CoreML when fix_gamma=True (#13557)
• Fix for test always returning true (#13911)
• Add error checking for cpp examples. (#13828)
• julia: fix argmax for NDArray (#13871)
• test_ImageRecordIter_seed_augmentation flaky test fix (#12485)
• Julia: fix filename quoting in docstring (#13894)
• Flaky maven binary download (#13974)
• MXNET-1293 Adding Iterables instead of List to method signature for infer APIs in Java (#13977)
• Sample python bilinear initializer at integral points in y-direction (#12983)
• Fix inconsistent handling for FResourceRequestEx for imperative and symbolic executor (#14007)
• MXNET-1258 fix unittest for ROIAlign Operator (#13609)
• Fix performance regression in normalize operator (#14055)
• Remove inplace support for ToTensor operator (#14083)
• Addresses comments in runtime feature discovery API (#13964)
• The latest version of leiningen has a dependency problem with codox (#14132)
• Fix quote on LBSGD docs (#13975)
• Fixes spelling (#14168)
• Fix broken amalgamation (#12792)
• Fix nd.pick large array issue (#14082)
• Fix req=null in SliceLikeBackward (#14209)
• onnx broadcast ops fixes (#13604)
• fix update params (#14218)
• MXNet Java bug fixes and experience improvement (#14213)
• reverting broadcasting fixes (#14299)
• fix memory-related issues to enable ASAN tests (#14223)
• FIX: flaky test exponential generator (#14287)
• fix SoftmaxOutput resource bug (#14302)
• Fix shape inference pass (#14153)
• Limit workspace for cudnnGet results (#14326)
• #14199: catch subprocess.CalledProcessError in get_gpus() (#14212)
• Fixes #14181, validate model output shape for ObjectDetector. (#14215)
• Optimizer MXKVStoreUpdater bug fix in serializeState method (#14337)
• Add proper exception message for negative shape in array creation routines (#14362)
• Fix NaN value comparisons in relu, max and min ops (#14262)
• fix engine crash in shutdown phase (#14382)
• Flaky test #14189 (#14190)
• Correct update count with Gluon trainer and update_on_kvstore=False (#14377)
• Fix relative difference scala (#14417)
• fix OOM error during resource allocation (#14444)
• Fix crashes on visualization (#14425)
• begin=end not a valid input (#14403)
• Fix memory leak for size-zero ndarray (#14365)
• Fixes the test_sgld (#14473)
• Revert "Fix memory leak for size-zero ndarray (#14365)" (#14477)
• fix custom operation in fork (#14451)
• Fixes test_operator_gpu.test_multinomial_generator (#14475)
• Fix script retrieval (#14519)
• Memory fixes. Resolves #10867, and resolves #14080 (#14372)
• Chouffe/clojure fix tests (#14531)
• clojure add draw-bounding-box interop (#14533)
• fix tests (#14565)
• Do not touch GPU 0 during ReleaseAll (#14550)
• MXNET-1357 Fix the cpp-examples to add exception handling (#14441)
• fix build cpp examples option (#14562)
• Fix flaky test poisson generator & test_negative_binomial_generator (#14571)
• Fixing unintentional variable overloading (#14438)
• fix quantize graph pass (#14605)
• replace std::random_shuffle to std::shuffle (#14523)
• Add exception handling support for waitall (#14397)
• split_and_load can now handle num_ctx > num_data. Issue #13909 (#14607)
• Fix aspect ratio sampling for RandomResizedCrop (#14585)
• MXNET-400 support string type for kvstore key in cpp-package (#10792)
• Fix warning on macro expansion using defined. (#14598)
• Fix scaladoc scalastyle violations in Infer package (#14671)
• Fix profiler check (#14677)
• Tweak the copy for the cudnn autotuning warning. (#14680)
• Properly handling custom op exception by modify engine (#14693)
• Disable USE_GPERFTOOLS (#14711)
• Reference engine from chunk via weak pointer (#14591)
• C++ fix type inconsistent issue when loading quantized parameters (#15038)
• Fix crash in random.shuffle operator (#15041)
• MXNET-1406 Fix DLManagedTensor deleter (#15016)
• Fixes lint issue in AMP (#15015)
• Fixed issue where the estimator was printing beyond the dataset size … (#14464)
• Fixes cuDNN version for CUDA 9.0 build environment (#15001)
• Fix the incorrect MKLDNN/MKL logic in cmake (#14877)
• Fixed and re-enables TensorRT steps (#14960)
• Fix the return type of sparse.clip operator (#14856)
• Fix sample_multinomial number of outputs bug (#14873)
• MXNET-13578 Fix cmake installation failed (#14692)
• Fix iterator over symbol when multiple children have the same name (#14597)
• Fixes for wine detection tutorial (#13886)
• Scala/Java Predict API fix #14756 (#14804)
• Fix GELU backward possible NaN (#14782)
• fix shape index bug (#14518)
• BUGFIX fix ELU function will appear nan when calculating the gradient (#14673)
• Change size_t to int within for loop to fix windows build error (#14740)
• contrib fix MultiBoxPrior confusing results if first ratio is not 1.0 (#13763)
• Fix scalastyle (#14669)
• fix Makefile (#14424)
• v1.4.x Update MKL-DNN to fix the OSX build issue (#14141) (#14182)
• add preprocessed data and pretrained model info; minor format/spelling fixes (#14170)
• Fixes libjpeg-turbo dependecy under Ubuntu 16.04 (#14127)
• Fix website error pages (#13963)
• fix Makefile for rpkg (#13590)
• fix c complier to clang (#13778)
• Fix #13521 (#13537)
• MXNET-1234 Fix shape inference problems in Activation backward (#13409)
• Revert the change broadcast_to param shape (#14998)
• Fix infer shape partial after unknown shape changed to -1 (#14869)
• fix add_n bug: when input mem overlap with output mem, results is wrong (#14889)
• Bugfix Fix layer norm for large input shape (#14870)
• Fix Clojure BERT example's context argument (#14843)
• fix min max on zero-sized ndarray (#14745)
• fix acc_type_switch macro with extra tests (#14773)
• fix bug in profiler tutorial when using cpu (#13695)
• MXNET-1291 solve pylint errors in examples with issue no.12205 (#13815)
• data preparation file moved in example (#14781)
• MXNET-1291 solve pylint errors in examples with issue no.12205 (#13848)
• Prevent crashes for opencv exception and std::exception (#14433)
• Set idx2name for Optimizer object (#14703)
• Revert "Bumped minor version from 1.4.0 to 1.5.0 on master, updated License file" (#13558)
• BUGFIX fix unknown parameter shapes when np_shape is turned on. (#15097)
• Add gluonCV to fix AMP Tutorial (#15039)
• fix the if condition for LayerNorm (#15094)
• MKLDNNFix mkldnn deconvolution forward with bias (#15088)
• NER example: fix divisions by zero (#15068)
• remove warning in tutorial: (#15135)
• MXNET-1291 solve pylint errors in examples with issue no.12205 (#13938)
• Revert "Improve cached_op performance for static mode (#14785)" (#14868)
• Fix mkldnn backend when using naive engine (#15089)
• fix gluon rnn cell single step unroll (#15081)
• Revert "Improve FC perf when no_bias=False (#15033)" (#15099)

License

• Updates python setup.py for recent license changes (#14778)
• MXNET-1377 Add static-dependencies licenses (#14726)
• add license (#13793)
• License update (#13565)
• Bumped minor version from 1.4.0 to 1.5.0 on master, updated License file (#13478)
• License Googletest and Appendix (#14687)
• Add copyrights for third party licenses to license file (#13851)
• Improve license_header tool by only traversing files under revision c… (#13803)
• Update LICENSE File with subcomponents (#13808)

Depreciations

• Julia: deprecate mx.empty, replace it with UndefInitializer (#13934)
• Deprecate NDArrayCollector and instead use ResourceScope (#14780)

Known Issues

• Amalgamation compile problems(#14808)
• Dynamic Shape does not support reverse shape inference and deferred initialization. (#14983)
• Disables flaky test_random_size_crop (#15019)
• Disables flaky test_l2_normalization (#15006)
• Disables flaky TestStochasticTiming_2D test (#14412)
• Disables flaky test_operator.test_sgld test (#14410)
• Disables test_bulking due to flakyness (#14971)
• Disabled flaky test (#13758)
• Disables flaky test_droupout (#15003)
• Disables flaky test_operator_gpu.test_activation (#14969)

Added

• Tokenization Added NGramTokenizer (#350)
• editorconfig file (#355)

Fixed

• Dataset FilesDataset read samples without additional array (#363)
• Tokenization fixed error with numeric token values (#363)

Changed

• Math improved performance with pow and sqrt replacement (#350)
• Math reduce duplicated code in distance metrics (#348)
• update phpunit to 7.5.1 (#335)
• code style fixes (#334)

This release of the Microsoft Cognitive Toolkit (CNTK) supports both Windows and Linux on 64bit platforms.

This release contains versions for both GPU and CPU-only systems.

See Release Notes or download them as a separate file using the link below.

To set up this release on your system, please read the Install instructions in the root of the corresponding archive and Binary Setup instructions in CNTK Wiki.

Windows

• CNTK for Windows v.2.7 CPU only
• CNTK for Windows v.2.7 GPU
• NuGet Packages:
  • CNTK, CPU-Only Build
  • CNTK, GPU Build
  • CNTK, UWP CPU-Only Build
  • CNTK CPU-only Model Evaluation Libraries (MKL based) Package description and instructions are here.

Linux

• CNTK for Linux v.2.7 CPU only
• CNTK for Linux v.2.7 GPU Images at Docker Hub are coming soon

Core

• Improvements
  • Continue moving the worker code to C++. #5031, #4966, #4922, #4899, #5032, #4996, #4875
  • Add a hash table data structure to the Redis modules. #4911
  • Use gRPC for communication between node managers. #4968, #5023, #5024
• Python
  • @ray.remote now inherits the function docstring. #4985
  • Remove typing module from setup.py install_requirements. #4971
• Java
  • Allow users to set JVM options at actor creation time. #4970
• Internal
  • Refactor IDs: DriverID -> JobID, change all ID functions to camel case. #4964, #4896
  • Improve organization of directory structure. #4898
• Peformance
  • Get task object dependencies in parallel from object store. #4775
  • Flush lineage cache on task submission instead of execution. #4942
  • Remove debug check for uncommitted lineage. #5038

Tune

• Add directional metrics for components. #4120, #4915
• Disallow setting resources_per_trial when it is already configured. #4880
• Make PBT Quantile fraction configurable. #4912

RLlib

• Add QMIX mixer parameters to optimizer param list. #5014
• Allow Torch policies access to full action input dict in extra_action_out_fn. #4894
• Allow access to batches prior to postprocessing. #4871
• Throw error if sample_async is used with pytorch for A3C. #5000
• Patterns & User Experience
  • Rename PolicyEvaluator => RolloutWorker. #4820
  • Port remainder of algorithms to build_trainer() pattern. #4920
  • Port DQN to build_tf_policy() pattern. #4823
• Documentation
  • Add docs on how to use TF eager execution. #4927
  • Add preprocessing example to offline documentation. #4950

Other Libraries

• Add support for distributed training with PyTorch. #4797, #4933
• Autoscaler will kill workers on exception. #4997
• Fix handling of non-integral timeout values in signal.receive. #5002

Thanks

We thank the following contributors for their amazing contributions: @jiangzihao2009, @raulchen, @ericl, @hershg, @kfstorm, @kiddyboots216, @jovany-wang, @pschafhalter, @richardliaw, @robertnishihara, @stephanie-wang, @simon-mo, @zhijunfu, @ls-daniel, @ajgokhale, @rueberger, @suquark, @guoyuhong, @jovany-wang, @pcmoritz, @hartikainen, @timonbimon, @TianhongDai

Release 1.12.3

Bug Fixes and Other Changes

• Updates png_archive dependency to 1.6.37 to not be affected by CVE-2019-7317, CVE-2018-13785, and CVE-2018-14048.
• Updates sqlite depenency to 3.28.0 to not be affected by CVE-2018-20506, CVE-2018-20346, and CVE-2018-20505.