diff --git a/docs/build/html/.buildinfo b/docs/build/html/.buildinfo new file mode 100644 index 000000000..39cf7df46 --- /dev/null +++ b/docs/build/html/.buildinfo @@ -0,0 +1,4 @@ +# Sphinx build info version 1 +# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. +config: 5b1e2769941ea186df86318a9fdcef20 +tags: 645f666f9bcd5a90fca523b33c5a78b7 diff --git a/docs/build/html/_sources/cpp/ops.rst.txt b/docs/build/html/_sources/cpp/ops.rst.txt new file mode 100644 index 000000000..4d2d1404e --- /dev/null +++ b/docs/build/html/_sources/cpp/ops.rst.txt @@ -0,0 +1,6 @@ +.. _cpp_ops: + +Operations +========== + + diff --git a/docs/build/html/_sources/dev/extensions.rst.txt b/docs/build/html/_sources/dev/extensions.rst.txt new file mode 100644 index 000000000..36f44a4bb --- /dev/null +++ b/docs/build/html/_sources/dev/extensions.rst.txt @@ -0,0 +1,948 @@ +Developer Documentation +======================= + +MLX provides a open and flexible backend to which users may add operations +and specialized implementations without much hassle. While the library supplies +efficient operations that can be used and composed for any number of +applications, there may arise cases where new functionalities or highly +optimized implementations are needed. For such cases, you may design and +implement your own operations that link to and build on top of :mod:`mlx.core`. +We will introduce the inner-workings of MLX and go over a simple example to +learn the steps involved in adding new operations to MLX with your own CPU +and GPU implementations. + +Introducing the Example +----------------------- + +Let's say that you would like an operation that takes in two arrays, +``x`` and ``y``, scales them both by some coefficents ``alpha`` and ``beta`` +respectively, and then adds them together to get the result +``z = alpha * x + beta * y``. Well, you can very easily do that by just +writing out a function as follows: + +.. code-block:: python + + import mlx.core as mx + + def simple_axpby(x: mx.array, y: mx.array, alpha: float, beta: float) -> mx.array: + return alpha * x + beta * y + +This function performs that operation while leaving the implementations and +differentiation to MLX. + +However, you work with vector math libraries often and realize that the +``axpby`` routine defines the same operation ``Y = (alpha * X) + (beta * Y)``. +You would really like the part of your applications that does this operation +on the CPU to be very fast - so you decide that you want it to rely on the +``axpby`` routine provided by the Accelerate_ framework. Continuing to impose +our assumptions on to you, let's also assume that you want to learn how add +your own implementation for the gradients of your new operation while going +over the ins-and-outs of the MLX framework. + +Well, what a coincidence! You are in the right place. Over the course of this +example, we will learn: + +* The structure of the MLX library from the frontend API to the backend implementations. +* How to implement your own CPU backend that redirects to Accelerate_ when appropriate (and a fallback if needed). +* How to implement your own GPU implementation using metal. +* How to add your own ``vjp`` and ``jvp``. +* How to build your implementations, link them to MLX, and bind them to python. + +Operations and Primitives +------------------------- + +In one sentence, operations in MLX build the computation graph, and primitives +provide the rules for evaluation and transformations of said graph. Let's start +by discussing operations in more detail. + +Operations +^^^^^^^^^^^ + +Operations are the frontend functions that operate on arrays. They are defined +in the C++ API (:ref:`cpp_ops`) and then we provide bindings to these +operations in the Python API (:ref:`ops`). + +We would like an operation, :meth:`axpby` that takes in two arrays ``x`` and ``y``, +and two scalars, ``alpha`` and ``beta``. This is how we would define it in the +C++ API: + +.. code-block:: C++ + + /** + * Scale and sum two vectors elementwise + * z = alpha * x + beta * y + * + * Follow numpy style broadcasting between x and y + * Inputs are upcasted to floats if needed + **/ + array axpby( + const array& x, // Input array x + const array& y, // Input array y + const float alpha, // Scaling factor for x + const float beta, // Scaling factor for y + StreamOrDevice s = {} // Stream on which to schedule the operation + ); + + +This operation itself can call other operations within it if needed. So, the +simplest way to go about implementing this operation would be do so in terms +of existing operations. + +.. code-block:: C++ + + array axpby( + const array& x, // Input array x + const array& y, // Input array y + const float alpha, // Scaling factor for x + const float beta, // Scaling factor for y + StreamOrDevice s /* = {} */ // Stream on which to schedule the operation + ) { + // Scale x and y on the provided stream + auto ax = multiply(array(alpha), x, s); + auto by = multiply(array(beta), y, s); + + // Add and return + return add(ax, by, s); + } + +However, as we discussed earlier, this is not our goal. The operations themselves +do not contain the implementations that act on the data, nor do they contain the +rules of transformations. Rather, they are an easy to use interface that build +on top of the building blocks we call :class:`Primitive`. + +Primitives +^^^^^^^^^^^ + +A :class:`Primitive` is part of the computation graph of an :class:`array`. It +defines how to create an output given a set of input :class:`array` . Further, +a :class:`Primitive` is a class that contains rules on how it is evaluated +on the CPU or GPU, and how it acts under transformations such as ``vjp`` and +``jvp``. These words on their own can be a bit abstract, so lets take a step +back and go to our example to give ourselves a more concrete image. + +.. code-block:: C++ + + class Axpby : public Primitive { + public: + explicit Axpby(Stream stream, float alpha, float beta) + : Primitive(stream), alpha_(alpha), beta_(beta){}; + + /** + * A primitive must know how to evaluate itself on the CPU/GPU + * for the given inputs and populate the output array. + * + * To avoid unecessary allocations, the evaluation function + * is responsible for allocating space for the array. + */ + void eval_cpu(const std::vector& inputs, array& out) override; + void eval_gpu(const std::vector& inputs, array& out) override; + + /** The Jacobian-vector product. */ + array jvp( + const std::vector& primals, + const std::vector& tangents, + const std::vector& argnums) override; + + /** The vector-Jacobian product. */ + std::vector vjp( + const std::vector& primals, + const array& cotan, + const std::vector& argnums) override; + + /** + * The primitive must know how to vectorize itself accross + * the given axes. The output is a pair containing the array + * representing the vectorized computation and the axis which + * corresponds to the output vectorized dimension. + */ + std::pair vmap( + const std::vector& inputs, + const std::vector& axes) override; + + /** Print the primitive. */ + void print(std::ostream& os) override { + os << "Axpby"; + } + + /** Equivalence check **/ + bool is_equivalent(const Primitive& other) const override; + + private: + float alpha_; + float beta_; + + /** Fall back implementation for evaluation on CPU */ + void eval(const std::vector& inputs, array& out); + }; + +The :class:`Axpby` class derives from the base :class:`Primitive` class and +follows the above demonstrated interface. :class:`Axpby` treats ``alpha`` and +``beta`` as parameters. It then provides implementations of how the array ``out`` +is produced given ``inputs`` through :meth:`Axpby::eval_cpu` and +:meth:`Axpby::eval_gpu`. Further, it provides rules of transformations in +:meth:`Axpby::jvp`, :meth:`Axpby::vjp`, and :meth:`Axpby::vmap`. + +Using the Primitives +^^^^^^^^^^^^^^^^^^^^^ + +Operations can use this :class:`Primitive` to add a new :class:`array` to +the computation graph. An :class:`array` can be constructed by providing its +data type, shape, the :class:`Primitive` that computes it, and the +:class:`array` inputs that are passed to the primitive. + +Let's re-implement our operation now in terms of our :class:`Axpby` primitive. + +.. code-block:: C++ + + array axpby( + const array& x, // Input array x + const array& y, // Input array y + const float alpha, // Scaling factor for x + const float beta, // Scaling factor for y + StreamOrDevice s /* = {} */ // Stream on which to schedule the operation + ) { + // Promote dtypes between x and y as needed + auto promoted_dtype = promote_types(x.dtype(), y.dtype()); + + // Upcast to float32 for non-floating point inputs x and y + auto out_dtype = is_floating_point(promoted_dtype) + ? promoted_dtype + : promote_types(promoted_dtype, float32); + + // Cast x and y up to the determined dtype (on the same stream s) + auto x_casted = astype(x, out_dtype, s); + auto y_casted = astype(y, out_dtype, s); + + // Broadcast the shapes of x and y (on the same stream s) + auto broadcasted_inputs = broadcast_arrays({x_casted, y_casted}, s); + auto out_shape = broadcasted_inputs[0].shape(); + + // Construct the array as the output of the Axpby primitive + // with the broadcasted and upcasted arrays as inputs + return array( + /* const std::vector& shape = */ out_shape, + /* Dtype dtype = */ out_dtype, + /* std::unique_ptr primitive = */ + std::make_unique(to_stream(s), alpha, beta), + /* const std::vector& inputs = */ broadcasted_inputs); + } + + +This operation now handles the following: + +#. Upcast inputs and resolve the the output data type. +#. Broadcast the inputs and resolve the output shape. +#. Construct the primitive :class:`Axpby` using the given stream, ``alpha``, and ``beta``. +#. Construct the output :class:`array` using the primitive and the inputs. + +Implementing the Primitive +-------------------------- + +No computation happens when we call the operation alone. In effect, the +operation only builds the computation graph. When we evaluate the output +array, MLX schedules the execution of the computation graph, and calls +:meth:`Axpby::eval_cpu` or :meth:`Axpby::eval_gpu` depending on the +stream/device specified by the user. + +.. warning:: + When :meth:`Primitive::eval_cpu` or :meth:`Primitive::eval_gpu` are called, + no memory has been allocated for the output array. It falls on the implementation + of these functions to allocate memory as needed + +Implementing the CPU Backend +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Let's start by trying to implement a naive and generic version of +:meth:`Axpby::eval_cpu`. We declared this as a private member function of +:class:`Axpby` earlier called :meth:`Axpby::eval`. + +Our naive method will go over each element of the output array, find the +corresponding input elements of ``x`` and ``y`` and perform the operation +pointwise. This is captured in the templated function :meth:`axpby_impl`. + +.. code-block:: C++ + + template + void axpby_impl( + const array& x, + const array& y, + array& out, + float alpha_, + float beta_) { + // We only allocate memory when we are ready to fill the output + // malloc_or_wait synchronously allocates available memory + // There may be a wait executed here if the allocation is requested + // under memory-pressured conditions + out.set_data(allocator::malloc_or_wait(out.nbytes())); + + // Collect input and output data pointers + const T* x_ptr = x.data(); + const T* y_ptr = y.data(); + T* out_ptr = out.data(); + + // Cast alpha and beta to the relevant types + T alpha = static_cast(alpha_); + T beta = static_cast(beta_); + + // Do the elementwise operation for each output + for (size_t out_idx = 0; out_idx < out.size(); out_idx++) { + // Map linear indices to offsets in x and y + auto x_offset = elem_to_loc(out_idx, x.shape(), x.strides()); + auto y_offset = elem_to_loc(out_idx, y.shape(), y.strides()); + + // We allocate the output to be contiguous and regularly strided + // (defaults to row major) and hence it doesn't need additonal mapping + out_ptr[out_idx] = alpha * x_ptr[x_offset] + beta * y_ptr[y_offset]; + } + } + +Now, we would like our implementation to be able to do this pointwise operation +for all incoming floating point arrays. Accordingly, we add dispatches for +``float32``, ``float16``, ``bfloat16`` and ``complex64``. We throw an error +if we encounter an unexpected type. + +.. code-block:: C++ + + /** Fall back implementation for evaluation on CPU */ + void Axpby::eval(const std::vector& inputs, array& out) { + // Check the inputs (registered in the op while contructing the out array) + assert(inputs.size() == 2); + auto& x = inputs[0]; + auto& y = inputs[1]; + + // Dispatch to the correct dtype + if (out.dtype() == float32) { + return axpby_impl(x, y, out, alpha_, beta_); + } else if (out.dtype() == float16) { + return axpby_impl(x, y, out, alpha_, beta_); + } else if (out.dtype() == bfloat16) { + return axpby_impl(x, y, out, alpha_, beta_); + } else if (out.dtype() == complex64) { + return axpby_impl(x, y, out, alpha_, beta_); + } else { + throw std::runtime_error( + "Axpby is only supported for floating point types."); + } + } + +We have a fallback implementation! Now, to do what we are really here to do. +Remember we wanted to use the ``axpby`` routine provided by the Accelerate_ +framework? Well, there are 3 complications to keep in mind: + +#. Accelerate does not provide implementations of ``axpby`` for half precision + floats. We can only direct to it for ``float32`` types +#. Accelerate assumes the inputs ``x`` and ``y`` are contiguous and all elements + have fixed strides between them. Possibly due to broadcasts and transposes, + we aren't guaranteed that the inputs fit this requirement. We can + only direct to Accelerate if both ``x`` and ``y`` are row contiguous or + column contiguous. +#. Accelerate performs the routine ``Y = (alpha * X) + (beta * Y)`` inplace. + MLX expects to write out the answer to a new array. We must copy the elements + of ``y`` into the output array and use that as an input to ``axpby`` + +Let's write out an implementation that uses Accelerate in the right conditions. +It must simply allocate data for the output, copy elements of ``y`` into it, +and then call the :meth:`catlas_saxpby` from accelerate. + +.. code-block:: C++ + + template + void axpby_impl_accelerate( + const array& x, + const array& y, + array& out, + float alpha_, + float beta_) { + // Accelerate library provides catlas_saxpby which does + // Y = (alpha * X) + (beta * Y) in place + // To use it, we first copy the data in y over to the output array + + // This specialization requires both x and y be contiguous in the same mode + // i.e: corresponding linear indices in both point to corresponding elements + // The data in the output array is allocated to match the strides in y + // such that x, y, and out are contiguous in the same mode and + // no transposition is needed + out.set_data( + allocator::malloc_or_wait(y.data_size() * out.itemsize()), + y.data_size(), + y.strides(), + y.flags()); + + // We then copy over the elements using the contiguous vector specialization + copy_inplace(y, out, CopyType::Vector); + + // Get x and y pointers for catlas_saxpby + const T* x_ptr = x.data(); + T* y_ptr = out.data(); + + T alpha = static_cast(alpha_); + T beta = static_cast(beta_); + + // Call the inplace accelerate operator + catlas_saxpby( + /* N = */ out.size(), + /* ALPHA = */ alpha, + /* X = */ x_ptr, + /* INCX = */ 1, + /* BETA = */ beta, + /* Y = */ y_ptr, + /* INCY = */ 1); + } + +Great! But what about the inputs that do not fit the criteria for accelerate? +Luckily, we can always just direct back to :meth:`Axpby::eval`. + +With this in mind, lets finally implement our :meth:`Axpby::eval_cpu`. + +.. code-block:: C++ + + /** Evaluate primitive on CPU using accelerate specializations */ + void Axpby::eval_cpu(const std::vector& inputs, array& out) { + assert(inputs.size() == 2); + auto& x = inputs[0]; + auto& y = inputs[1]; + + // Accelerate specialization for contiguous single precision float arrays + if (out.dtype() == float32 && + ((x.flags().row_contiguous && y.flags().row_contiguous) || + (x.flags().col_contiguous && y.flags().col_contiguous))) { + axpby_impl_accelerate(x, y, out, alpha_, beta_); + return; + } + + // Fall back to common backend if specializations are not available + eval(inputs, out); + } + +We have now hit a milestone! Just this much is enough to run the operation +:meth:`axpby` on a CPU stream! + +If you do not plan on running the operation on the GPU or using transforms on +computation graphs that contain :class:`Axpby`, you can stop implementing the +primitive here and enjoy the speed-ups you get from the Accelerate library. + +Implementing the GPU Backend +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Apple silicon devices address their GPUs using the Metal_ shading language, and +all GPU kernels in MLX are written using metal. + +.. note:: + + Here are some helpful resources if you are new to metal! + + * A walkthrough of the metal compute pipeline: `Metal Example`_ + * Documentation for metal shading language: `Metal Specification`_ + * Using metal from C++: `Metal-cpp`_ + +Let's keep the GPU algorithm simple. We will launch exactly as many threads +as there are elements in the output. Each thread will pick the element it needs +from ``x`` and ``y``, do the pointwise operation, and then update its assigned +element in the output. + +.. code-block:: C++ + + template + [[kernel]] void axpby_general( + device const T* x [[buffer(0)]], + device const T* y [[buffer(1)]], + device T* out [[buffer(2)]], + constant const float& alpha [[buffer(3)]], + constant const float& beta [[buffer(4)]], + constant const int* shape [[buffer(5)]], + constant const size_t* x_strides [[buffer(6)]], + constant const size_t* y_strides [[buffer(7)]], + constant const int& ndim [[buffer(8)]], + uint index [[thread_position_in_grid]]) { + // Convert linear indices to offsets in array + auto x_offset = elem_to_loc(index, shape, x_strides, ndim); + auto y_offset = elem_to_loc(index, shape, y_strides, ndim); + + // Do the operation and update the output + out[index] = + static_cast(alpha) * x[x_offset] + static_cast(beta) * y[y_offset]; + } + +We then need to instantiate this template for all floating point types and give +each instantiation a unique host name so we can identify the right kernel for +each data type. + +.. code-block:: C++ + + #define instantiate_axpby(type_name, type) \ + template [[host_name("axpby_general_" #type_name)]] \ + [[kernel]] void axpby_general( \ + device const type* x [[buffer(0)]], \ + device const type* y [[buffer(1)]], \ + device type* out [[buffer(2)]], \ + constant const float& alpha [[buffer(3)]], \ + constant const float& beta [[buffer(4)]], \ + constant const int* shape [[buffer(5)]], \ + constant const size_t* x_strides [[buffer(6)]], \ + constant const size_t* y_strides [[buffer(7)]], \ + constant const int& ndim [[buffer(8)]], \ + uint index [[thread_position_in_grid]]); + + instantiate_axpby(float32, float); + instantiate_axpby(float16, half); + instantiate_axpby(bflot16, bfloat16_t); + instantiate_axpby(complex64, complex64_t); + +This kernel will be compiled into a metal library ``mlx_ext.metallib`` as we +will see later in :ref:`Building with CMake`. In the following example, we +assume that the library ``mlx_ext.metallib`` will always be co-located with +the executable/ shared-library calling the :meth:`register_library` function. +The :meth:`register_library` function takes the library's name and potential +path (or in this case, a function that can produce the path of the metal +library) and tries to load that library if it hasn't already been registered +by the relevant static :class:`mlx::core::metal::Device` object. This is why, +it is important to package your C++ library with the metal library. We will +go over this process in more detail later. + +The logic to determine the kernel, set the inputs, resolve the grid dimensions +and dispatch it to the GPU are contained in :meth:`Axpby::eval_gpu` as shown +below. + +.. code-block:: C++ + + /** Evaluate primitive on GPU */ + void Axpby::eval_gpu(const std::vector& inputs, array& out) { + // Prepare inputs + assert(inputs.size() == 2); + auto& x = inputs[0]; + auto& y = inputs[1]; + + // Each primitive carries the stream it should execute on + // and each stream carries its device identifiers + auto& s = stream(); + // We get the needed metal device using the stream + auto& d = metal::device(s.device); + + // Allocate output memory + out.set_data(allocator::malloc_or_wait(out.nbytes())); + + // Resolve name of kernel (corresponds to axpby.metal) + std::ostringstream kname; + kname << "axpby_" << "general_" << type_to_name(out); + + // Make sure the metal library is available and look for it + // in the same folder as this executable if needed + d.register_library("mlx_ext", metal::get_colocated_mtllib_path); + + // Make a kernel from this metal library + auto kernel = d.get_kernel(kname.str(), "mlx_ext"); + + // Prepare to encode kernel + auto compute_encoder = d.get_command_encoder(s.index); + compute_encoder->setComputePipelineState(kernel); + + // Kernel parameters are registered with buffer indices corresponding to + // those in the kernel decelaration at axpby.metal + int ndim = out.ndim(); + size_t nelem = out.size(); + + // Encode input arrays to kernel + set_array_buffer(compute_encoder, x, 0); + set_array_buffer(compute_encoder, y, 1); + + // Encode output arrays to kernel + set_array_buffer(compute_encoder, out, 2); + + // Encode alpha and beta + compute_encoder->setBytes(&alpha_, sizeof(float), 3); + compute_encoder->setBytes(&beta_, sizeof(float), 4); + + // Encode shape, strides and ndim + compute_encoder->setBytes(x.shape().data(), ndim * sizeof(int), 5); + compute_encoder->setBytes(x.strides().data(), ndim * sizeof(size_t), 6); + compute_encoder->setBytes(y.strides().data(), ndim * sizeof(size_t), 7); + compute_encoder->setBytes(&ndim, sizeof(int), 8); + + // We launch 1 thread for each input and make sure that the number of + // threads in any given threadgroup is not higher than the max allowed + size_t tgp_size = std::min(nelem, kernel->maxTotalThreadsPerThreadgroup()); + + // Fix the 3D size of each threadgroup (in terms of threads) + MTL::Size group_dims = MTL::Size(tgp_size, 1, 1); + + // Fix the 3D size of the launch grid (in terms of threads) + MTL::Size grid_dims = MTL::Size(nelem, 1, 1); + + // Launch the grid with the given number of threads divded among + // the given threadgroups + compute_encoder->dispatchThreads(grid_dims, group_dims); + } + +We can now call the :meth:`axpby` operation on both the CPU and the GPU! + +A few things to note about MLX and metal before moving on. MLX keeps track +of the active ``compute_encoder``. We rely on :meth:`d.get_command_encoder` +to give us the active metal compute command encoder instead of building a +new one and calling :meth:`compute_encoder->end_encoding` at the end. +MLX keeps adding kernels (compute pipelines) to the active command encoder +until some specified limit is hit or the compute encoder needs to be flushed +for synchronization. MLX also handles enqueuing and commiting the associated +command buffers as needed. We suggest taking a deeper dive into +:class:`metal::Device` if you would like to study this routine further. + +Primitive Transforms +^^^^^^^^^^^^^^^^^^^^^ + +Now that we have come this far, let's also learn how to add implementations to +transformations in a :class:`Primitive`. These transformations can be built on +top of our operations, including the one we just defined now. Which then gives +us the following :meth:`Axpby::jvp` and :meth:`Axpby::vjp` implementations. + +.. code-block:: C++ + + /** The Jacobian-vector product. */ + array Axpby::jvp( + const std::vector& primals, + const std::vector& tangents, + const std::vector& argnums) { + // Forward mode diff that pushes along the tangents + // The jvp transform on the the primitive can built with ops + // that are scheduled on the same stream as the primtive + + // If argnums = {0}, we only push along x in which case the + // jvp is just the tangent scaled by alpha + // Similarly, if argnums = {1}, the jvp is just the tangent + // scaled by beta + if (argnums.size() > 1) { + auto scale = argnums[0] == 0 ? alpha_ : beta_; + auto scale_arr = array(scale, tangents[0].dtype()); + return multiply(scale_arr, tangents[0], stream()); + } + // If, argnums = {0, 1}, we take contributions from both + // which gives us jvp = tangent_x * alpha + tangent_y * beta + else { + return axpby(tangents[0], tangents[1], alpha_, beta_, stream()); + } + } + +.. code-block:: C++ + + /** The vector-Jacobian product. */ + std::vector Axpby::vjp( + const std::vector& primals, + const array& cotan, + const std::vector& argnums) { + // Reverse mode diff + std::vector vjps; + for (auto arg : argnums) { + auto scale = arg == 0 ? alpha_ : beta_; + auto scale_arr = array(scale, cotan.dtype()); + vjps.push_back(multiply(scale_arr, cotan, stream())); + } + return vjps; + } + +Finally, you need not have a transformation fully defined to start using your +own :class:`Primitive`. + +.. code-block:: C++ + + /** Vectorize primitve along given axis */ + std::pair Axpby::vmap( + const std::vector& inputs, + const std::vector& axes) { + throw std::runtime_error("Axpby has no vmap implementation."); + } + +Building and Binding +-------------------- + +Let's look at the overall directory structure first. + +| extensions +| ├── axpby +| │ ├── axpby.cpp +| │ ├── axpby.h +| │ └── axpby.metal +| ├── mlx_sample_extensions +| │ └── __init__.py +| ├── bindings.cpp +| ├── CMakeLists.txt +| └── setup.py + +* ``extensions/axpby/`` defines the C++ extension library +* ``extensions/mlx_sample_extensions`` sets out the strucutre for the + associated python package +* ``extensions/bindings.cpp`` provides python bindings for our operation +* ``extensions/CMakeLists.txt`` holds CMake rules to build the library and + python bindings +* ``extensions/setup.py`` holds the ``setuptools`` rules to build and install + the python package + +Binding to Python +^^^^^^^^^^^^^^^^^^ + +We use PyBind11_ to build a Python API for the C++ library. Since bindings +for all needed components such as `mlx.core.array`, `mlx.core.stream`, etc. +are already provided, adding our :meth:`axpby` becomes very simple! + +.. code-block:: C++ + + PYBIND11_MODULE(mlx_sample_extensions, m) { + m.doc() = "Sample C++ and metal extensions for MLX"; + + m.def( + "axpby", + &axpby, + "x"_a, + "y"_a, + py::pos_only(), + "alpha"_a, + "beta"_a, + py::kw_only(), + "stream"_a = py::none(), + R"pbdoc( + Scale and sum two vectors elementwise + ``z = alpha * x + beta * y`` + + Follows numpy style broadcasting between ``x`` and ``y`` + Inputs are upcasted to floats if needed + + Args: + x (array): Input array. + y (array): Input array. + alpha (float): Scaling factor for ``x``. + beta (float): Scaling factor for ``y``. + + Returns: + array: ``alpha * x + beta * y`` + )pbdoc"); + } + +Most of the complexity in the above example comes from additional bells and +whistles such as the literal names and doc-strings. + +.. warning:: + + :mod:`mlx.core` needs to be imported before importing + :mod:`mlx_sample_extensions` as defined by the pybind11 module above to + ensure that the casters for :mod:`mlx.core` components like + :class:`mlx.core.array` are available. + +.. _Building with CMake: + +Building with CMake +^^^^^^^^^^^^^^^^^^^^ + +Building the C++ extension library itself is simple, it only requires that you +``find_package(MLX CONFIG)`` and then link it to your library. + +.. code-block:: cmake + + # Add library + add_library(mlx_ext) + + # Add sources + target_sources( + mlx_ext + PUBLIC + ${CMAKE_CURRENT_LIST_DIR}/axpby/axpby.cpp + ) + + # Add include headers + target_include_directories( + mlx_ext PUBLIC ${CMAKE_CURRENT_LIST_DIR} + ) + + # Link to mlx + target_link_libraries(mlx_ext PUBLIC mlx) + +We also need to build the attached metal library. For convenience, we provide a +:meth:`mlx_build_metallib` function that builds a ``.metallib`` target given +sources, headers, destinations, etc. (defined in ``cmake/extension.cmake`` and +automatically imported with MLX package). + +Here is what that looks like in practice! + +.. code-block:: cmake + + # Build metallib + if(MLX_BUILD_METAL) + + mlx_build_metallib( + TARGET mlx_ext_metallib + TITLE mlx_ext + SOURCES ${CMAKE_CURRENT_LIST_DIR}/axpby/axpby.metal + INCLUDE_DIRS ${PROJECT_SOURCE_DIR} ${MLX_INCLUDE_DIRS} + OUTPUT_DIRECTORY ${CMAKE_LIBRARY_OUTPUT_DIRECTORY} + ) + + add_dependencies( + mlx_ext + mlx_ext_metallib + ) + + endif() + +Finally, we build the Pybind11_ bindings + +.. code-block:: cmake + + pybind11_add_module( + mlx_sample_extensions + ${CMAKE_CURRENT_LIST_DIR}/bindings.cpp + ) + target_link_libraries(mlx_sample_extensions PRIVATE mlx_ext) + + if(BUILD_SHARED_LIBS) + target_link_options(mlx_sample_extensions PRIVATE -Wl,-rpath,@loader_path) + endif() + +Building with ``setuptools`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Once we have set out the CMake build rules as described above, we can use the +build utilities defined in :mod:`mlx.extension` for a simple build process. + +.. code-block:: python + + from mlx import extension + from setuptools import setup + + if __name__ == "__main__": + setup( + name="mlx_sample_extensions", + version="0.0.0", + description="Sample C++ and Metal extensions for MLX primitives.", + ext_modules=[extension.CMakeExtension("mlx_sample_extensions")], + cmdclass={"build_ext": extension.CMakeBuild}, + packages = ["mlx_sample_extensions"], + package_dir = {"": "mlx_sample_extensions"}, + package_data = {"mlx_sample_extensions" : ["*.so", "*.dylib", "*.metallib"]}, + zip_safe=False, + python_requires=">=3.7", + ) + +.. note:: + We treat ``extensions/mlx_sample_extensions`` as the package directory + even though it only contains a ``__init__.py`` to ensure the following: + + * :mod:`mlx.core` is always imported before importing :mod:`mlx_sample_extensions` + * The C++ extension library and the metal library are co-located with the python + bindings and copied together if the package is installed + +You can build inplace for development using +``python setup.py build_ext -j8 --inplace`` (in ``extensions/``) + +This will result in a directory structure as follows: + +| extensions +| ├── mlx_sample_extensions +| │ ├── __init__.py +| │ ├── libmlx_ext.dylib # C++ extension library +| │ ├── mlx_ext.metallib # Metal library +| │ └── mlx_sample_extensions.cpython-3x-darwin.so # Python Binding +| ... + +When you try to install using the command ``python -m pip install .`` +(in ``extensions/``), the package will be installed with the same strucutre as +``extensions/mlx_sample_extensions`` and the C++ and metal library will be +copied along with the python binding since they are specified as ``package_data``. + +Usage +----- + +After installing the extension as described above, you should be able to simply +import the python package and play with it as you would any other MLX operation! + +Let's looks at a simple script and it's results! + +.. code-block:: python + + import mlx.core as mx + from mlx_sample_extensions import axpby + + a = mx.ones((3, 4)) + b = mx.ones((3, 4)) + c = axpby(a, b, 4.0, 2.0, stream=mx.cpu) + + print(f"c shape: {c.shape}") + print(f"c dtype: {c.dtype}") + print(f"c correctness: {mx.all(c == 6.0).item()}") + +Output: + +.. code-block:: + + c shape: [3, 4] + c dtype: float32 + c correctness: True + +Results +^^^^^^^^^^^^^^^^ + +Let's run a quick benchmark and see how our new ``axpby`` operation compares +with the naive :meth:`simple_axpby` we defined at first on the CPU. + +.. code-block:: python + + import mlx.core as mx + from mlx_sample_extensions import axpby + import time + + mx.set_default_device(mx.cpu) + + def simple_axpby(x: mx.array, y: mx.array, alpha: float, beta: float) -> mx.array: + return alpha * x + beta * y + + M = 256 + N = 512 + + x = mx.random.normal((M, N)) + y = mx.random.normal((M, N)) + alpha = 4.0 + beta = 2.0 + + mx.eval((x, y)) + + def bench(f): + # Warm up + for i in range(100): + z = f(x, y, alpha, beta) + mx.eval(z) + + # Timed run + s = time.time() + for i in range(5000): + z = f(x, y, alpha, beta) + mx.eval(z) + e = time.time() + return e - s + + simple_time = bench(simple_axpby) + custom_time = bench(axpby) + + print(f"Simple axpby: {simple_time:.3f} s | Custom axpby: {custom_time:.3f} s") + +Results: + +.. code-block:: + + Simple axpby: 0.114 s | Custom axpby: 0.109 s + +We see some modest improvements right away! + +This operation is now good to be used to build other operations, +in :class:`mlx.nn.Module` calls, and also as a part of graph +transformations such as :meth:`grad` and :meth:`simplify`! + +Scripts +------- + +.. admonition:: Download the code + + The full example code is available in `mlx-examples `_. + +.. code: `TODO_LINK/extensions`_ + +.. _Accelerate: https://developer.apple.com/documentation/accelerate/blas?language=objc +.. _Metal: https://developer.apple.com/documentation/metal?language=objc +.. _Metal-cpp: https://developer.apple.com/metal/cpp/ +.. _`Metal Specification`: https://developer.apple.com/metal/Metal-Shading-Language-Specification.pdf +.. _`Metal Example`: https://developer.apple.com/documentation/metal/performing_calculations_on_a_gpu?language=objc +.. _PyBind11: https://pybind11.readthedocs.io/en/stable/ \ No newline at end of file diff --git a/docs/build/html/_sources/examples/linear_regression.rst.txt b/docs/build/html/_sources/examples/linear_regression.rst.txt new file mode 100644 index 000000000..19fc0d435 --- /dev/null +++ b/docs/build/html/_sources/examples/linear_regression.rst.txt @@ -0,0 +1,77 @@ +.. _linear_regression: + +Linear Regression +----------------- + +Let's implement a basic linear regression model as a starting point to +learn MLX. First import the core package and setup some problem metadata: + +.. code-block:: python + + import mlx.core as mx + + num_features = 100 + num_examples = 1_000 + num_iters = 10_000 # iterations of SGD + lr = 0.01 # learning rate for SGD + + +We'll generate a synthetic dataset by: + +1. Sampling the design matrix ``X``. +2. Sampling a ground truth parameter vector ``w_star``. +3. Compute the dependent values ``y`` by adding Gaussian noise to ``X @ w_star``. + +.. code-block:: python + + # True parameters + w_star = mx.random.normal((num_features,)) + + # Input examples (design matrix) + X = mx.random.normal((num_examples, num_features)) + + # Noisy labels + eps = 1e-2 * mx.random.normal((num_examples,)) + y = X @ w_star + eps + + +We will use SGD to find the optimal weights. To start, define the squared loss +and get the gradient function of the loss with respect to the parameters. + +.. code-block:: python + + def loss_fn(w): + return 0.5 * mx.mean(mx.square(X @ w - y)) + + grad_fn = mx.grad(loss_fn) + +Start the optimization by initializing the parameters ``w`` randomly. Then +repeatedly update the parameters for ``num_iters`` iterations. + +.. code-block:: python + + w = 1e-2 * mx.random.normal((num_features,)) + + for _ in range(num_iters): + grad = grad_fn(w) + w = w - lr * grad + mx.eval(w) + +Finally, compute the loss of the learned parameters and verify that they are +close to the ground truth parameters. + +.. code-block:: python + + loss = loss_fn(w) + error_norm = mx.sum(mx.square(w - w_star)).item() ** 0.5 + + print( + f"Loss {loss.item():.5f}, |w-w*| = {error_norm:.5f}, " + ) + # Should print something close to: Loss 0.00005, |w-w*| = 0.00364 + +Complete `linear regression +`_ +and `logistic regression +`_ +examples are available in the MLX GitHub repo. diff --git a/docs/build/html/_sources/examples/llama-inference.rst.txt b/docs/build/html/_sources/examples/llama-inference.rst.txt new file mode 100644 index 000000000..c1a6a7e84 --- /dev/null +++ b/docs/build/html/_sources/examples/llama-inference.rst.txt @@ -0,0 +1,382 @@ +LLM inference +============== + +MLX enables efficient inference of large-ish transformers on Apple silicon +without compromising on ease of use. In this example we will create an +inference script for the Llama family of transformer models in which the model +is defined in less than 200 lines of python. + +Implementing the model +---------------------- + +We will use the neural network building blocks defined in the :mod:`mlx.nn` +module to concisely define the model architecture. + +Attention layer +^^^^^^^^^^^^^^^^ + +We will start with the llama attention layer which notably uses the RoPE +positional encoding. [1]_ In addition, our attention layer will optionally use a +key/value cache that will be concatenated with the provided keys and values to +support efficient inference. + +Our implementation uses :class:`mlx.nn.Linear` for all the projections and +:class:`mlx.nn.RoPE` for the positional encoding. + +.. code-block:: python + + import mlx.core as mx + import mlx.nn as nn + + class LlamaAttention(nn.Module): + def __init__(self, dims: int, num_heads: int): + super().__init__() + + self.num_heads = num_heads + + self.rope = nn.RoPE(dims // num_heads, traditional=True) + self.query_proj = nn.Linear(dims, dims, bias=False) + self.key_proj = nn.Linear(dims, dims, bias=False) + self.value_proj = nn.Linear(dims, dims, bias=False) + self.out_proj = nn.Linear(dims, dims, bias=False) + + def __call__(self, queries, keys, values, mask=None, cache=None): + queries = self.query_proj(queries) + keys = self.key_proj(keys) + values = self.value_proj(values) + + # Extract some shapes + num_heads = self.num_heads + B, L, D = queries.shape + + # Prepare the queries, keys and values for the attention computation + queries = queries.reshape(B, L, num_heads, -1).transpose(0, 2, 1, 3) + keys = keys.reshape(B, L, num_heads, -1).transpose(0, 2, 1, 3) + values = values.reshape(B, L, num_heads, -1).transpose(0, 2, 1, 3) + + # Add RoPE to the queries and keys and combine them with the cache + if cache is not None: + key_cache, value_cache = cache + queries = self.rope(queries, offset=key_cache.shape[2]) + keys = self.rope(keys, offset=key_cache.shape[2]) + keys = mx.concatenate([key_cache, keys], axis=2) + values = mx.concatenate([value_cache, values], axis=2) + else: + queries = self.rope(queries) + keys = self.rope(keys) + + # Finally perform the attention computation + scale = math.sqrt(1 / queries.shape[-1]) + scores = (queries * scale) @ keys.transpose(0, 1, 3, 2) + if mask is not None: + scores = scores + mask + scores = mx.softmax(scores, axis=-1) + values_hat = (scores @ values).transpose(0, 2, 1, 3).reshape(B, L, -1) + + # Note that we return the keys and values to possibly be used as a cache + return self.out_proj(values_hat), (keys, values) + +Encoder layer +^^^^^^^^^^^^^ + +The other component of the Llama model is the encoder layer which uses RMS +normalization [2]_ and SwiGLU. [3]_ For RMS normalization we will use +:class:`mlx.nn.RMSNorm` that is already provided in :mod:`mlx.nn`. + +.. code-block:: python + + class LlamaEncoderLayer(nn.Module): + def __init__(self, dims: int, mlp_dims: int, num_heads: int): + super().__init__() + + self.attention = LlamaAttention(dims, num_heads) + + self.norm1 = nn.RMSNorm(dims) + self.norm2 = nn.RMSNorm(dims) + + self.linear1 = nn.Linear(dims, mlp_dims, bias=False) + self.linear2 = nn.Linear(dims, mlp_dims, bias=False) + self.linear3 = nn.Linear(mlp_dims, dims, bias=False) + + def __call__(self, x, mask=None, cache=None): + y = self.norm1(x) + y, cache = self.attention(y, y, y, mask, cache) + x = x + y + + y = self.norm2(x) + a = self.linear1(y) + b = self.linear2(y) + y = a * mx.sigmoid(a) * b + y = self.linear3(y) + x = x + y + + return x, cache + +Full model +^^^^^^^^^^ + +To implement any Llama model we simply have to combine ``LlamaEncoderLayer`` +instances with an :class:`mlx.nn.Embedding` to embed the input tokens. + +.. code-block:: python + + class Llama(nn.Module): + def __init__( + self, num_layers: int, vocab_size: int, dims: int, mlp_dims: int, num_heads: int + ): + super().__init__() + + self.embedding = nn.Embedding(vocab_size, dims) + self.layers = [ + LlamaEncoderLayer(dims, mlp_dims, num_heads) for _ in range(num_layers) + ] + self.norm = nn.RMSNorm(dims) + self.out_proj = nn.Linear(dims, vocab_size, bias=False) + + def __call__(self, x): + mask = nn.MultiHeadAttention.create_additive_causal_mask(x.shape[1]) + mask = mask.astype(self.embedding.weight.dtype) + + x = self.embedding(x) + for l in self.layers: + x, _ = l(x, mask) + x = self.norm(x) + return self.out_proj(x) + +Note that in the implementation above we use a simple list to hold the encoder +layers but using ``model.parameters()`` will still consider these layers. + +Generation +^^^^^^^^^^^ + +Our ``Llama`` module can be used for training but not inference as the +``__call__`` method above processes one input, completely ignores the cache and +performs no sampling whatsoever. In the rest of this subsection, we will +implement the inference function as a python generator that processes the +prompt and then autoregressively yields tokens one at a time. + +.. code-block:: python + + class Llama(nn.Module): + ... + + def generate(self, x, temp=1.0): + cache = [] + + # Make an additive causal mask. We will need that to process the prompt. + mask = nn.MultiHeadAttention.create_additive_causal_mask(x.shape[1]) + mask = mask.astype(self.embedding.weight.dtype) + + # First we process the prompt x the same way as in __call__ but + # save the caches in cache + x = self.embedding(x) + for l in self.layers: + x, c = l(x, mask=mask) + cache.append(c) # <--- we store the per layer cache in a + # simple python list + x = self.norm(x) + y = self.out_proj(x[:, -1]) # <--- we only care about the last logits + # that generate the next token + y = mx.random.categorical(y * (1/temp)) + + # y now has size [1] + # Since MLX is lazily evaluated nothing is computed yet. + # Calling y.item() would force the computation to happen at + # this point but we can also choose not to do that and let the + # user choose when to start the computation. + yield y + + # Now we parsed the prompt and generated the first token we + # need to feed it back into the model and loop to generate the + # rest. + while True: + # Unsqueezing the last dimension to add a sequence length + # dimension of 1 + x = y[:, None] + + x = self.embedding(x) + for i in range(len(cache)): + # We are overwriting the arrays in the cache list. When + # the computation will happen, MLX will be discarding the + # old cache the moment it is not needed anymore. + x, cache[i] = self.layers[i](x, mask=None, cache=cache[i]) + x = self.norm(x) + y = self.out_proj(x[:, -1]) + y = mx.random.categorical(y * (1/temp)) + + yield y + +Putting it all together +^^^^^^^^^^^^^^^^^^^^^^^ + +We now have everything we need to create a Llama model and sample tokens from +it. In the following code, we randomly initialize a small Llama model, process +6 tokens of prompt and generate 10 tokens. + +.. code-block:: python + + model = Llama(num_layers=12, vocab_size=8192, dims=512, mlp_dims=1024, num_heads=8) + + # Since MLX is lazily evaluated nothing has actually been materialized yet. + # We could have set the `dims` to 20_000 on a machine with 8GB of RAM and the + # code above would still run. Let's actually materialize the model. + mx.eval(model.parameters()) + + prompt = mx.array([[1, 10, 8, 32, 44, 7]]) # <-- Note the double brackets because we + # have a batch dimension even + # though it is 1 in this case + + generated = [t for i, t in zip(range(10), model.generate(prompt, 0.8))] + + # Since we haven't evaluated anything, nothing is computed yet. The list + # `generated` contains the arrays that hold the computation graph for the + # full processing of the prompt and the generation of 10 tokens. + # + # We can evaluate them one at a time, or all together. Concatenate them or + # print them. They would all result in very similar runtimes and give exactly + # the same results. + mx.eval(generated) + +Converting the weights +---------------------- + +This section assumes that you have access to the original Llama weights and the +SentencePiece model that comes with them. We will write a small script to +convert the PyTorch weights to MLX compatible ones and write them in a NPZ file +that can be loaded directly by MLX. + +.. code-block:: python + + import argparse + from itertools import starmap + + import numpy as np + import torch + + def map_torch_to_mlx(key, value): + if "tok_embedding" in key: + key = "embedding.weight" + + elif "norm" in key: + key = key.replace("attention_norm", "norm1").replace("ffn_norm", "norm2") + + elif "wq" in key or "wk" in key or "wv" in key or "wo" in key: + key = key.replace("wq", "query_proj") + key = key.replace("wk", "key_proj") + key = key.replace("wv", "value_proj") + key = key.replace("wo", "out_proj") + + elif "w1" in key or "w2" in key or "w3" in key: + # The FFN is a separate submodule in PyTorch + key = key.replace("feed_forward.w1", "linear1") + key = key.replace("feed_forward.w3", "linear2") + key = key.replace("feed_forward.w2", "linear3") + + elif "output" in key: + key = key.replace("output", "out_proj") + + elif "rope" in key: + return None, None + + return key, value.numpy() + + + if __name__ == "__main__": + parser = argparse.ArgumentParser(description="Convert Llama weights to MLX") + parser.add_argument("torch_weights") + parser.add_argument("output_file") + args = parser.parse_args() + + state = torch.load(args.torch_weights) + np.savez( + args.output_file, + **{k: v for k, v in starmap(map_torch_to_mlx, state.items()) if k is not None} + ) + + +Weight loading and benchmarking +------------------------------- + +After converting the weights to be compatible to our implementation, all that is +left is to load them from disk and we can finally use the LLM to generate text. +We can load numpy format files using the :func:`mlx.core.load` operation. + +To create a parameter dictionary from the key/value representation of NPZ files +we will use the :func:`mlx.utils.tree_unflatten` helper method as follows: + +.. code-block:: python + + from mlx.utils import tree_unflatten + + model.update(tree_unflatten(list(mx.load(weight_file).items()))) + +:meth:`mlx.utils.tree_unflatten` will take keys from the NPZ file that look +like ``layers.2.attention.query_proj.weight`` and will transform them to + +.. code-block:: python + + {"layers": [..., ..., {"attention": {"query_proj": {"weight": ...}}}]} + +which can then be used to update the model. Note that the method above incurs +several unnecessary copies from disk to numpy and then from numpy to MLX. It +will be replaced in the future with direct loading to MLX. + +You can download the full example code in `mlx-examples `_. Assuming, the +existence of ``weights.pth`` and ``tokenizer.model`` in the current working +directory we can play around with our inference script as follows (the timings +are representative of an M1 Ultra and the 7B parameter Llama model): + +.. code-block:: bash + + $ python convert.py weights.pth llama-7B.mlx.npz + $ python llama.py llama-7B.mlx.npz tokenizer.model 'Call me Ishmael. Some years ago never mind how long precisely' + [INFO] Loading model from disk: 5.247 s + Press enter to start generation + ------ + , having little or no money in my purse, and nothing of greater consequence in my mind, I happened to be walking down Gower Street in the afternoon, in the heavy rain, and I saw a few steps off, a man in rags, who sat upon his bundle and looked hard into the wet as if he were going to cry. I watched him attentively for some time, and could not but observe that, though a numerous crowd was hurrying up and down, + ------ + [INFO] Prompt processing: 0.437 s + [INFO] Full generation: 4.330 s + +We observe that 4.3 seconds are required to generate 100 tokens and 0.4 seconds +of those are spent processing the prompt. This amounts to a little over **39 ms +per token**. + +By running with a much bigger prompt we can see that the per token generation +time as well as the prompt processing time remains almost constant. + +.. code-block:: bash + + $ python llama.py llama-7B.mlx.npz tokenizer.model 'Call me Ishmael. Some years ago never mind how long precisely, having little or no money in my purse, and nothing of greater consequence in my mind, I happened to be walking down Gower Street in the afternoon, in the heavy rain, and I saw a few steps off, a man in rags, who sat upon his bundle and looked hard into the wet as if he were going to cry. I watched him attentively for some time, and could not but observe that, though a numerous crowd was hurrying up and down, nobody took the least notice of him. I stopped at last, at a little distance, as if I had been in doubt, and after looking on a few minutes, walked straight up to him. He slowly raised his eyes, and fixed them upon me for a moment, without speaking, and then resumed his place and posture as before. I stood looking at him for a while, feeling very much pain at heart, and then said to him, “What are you doing there?” Something like a smile passed over his face, as he said slowly, “I am waiting for someone; but it has been three quarters of an hour now, and he has not come.” “What is it you are waiting for?” said I. Still he made no immediate reply, but again put his face down upon his hands, and did not' + [INFO] Loading model from disk: 5.247 s + Press enter to start generation + ------ + take his eyes from the ground. “What is it you are waiting for?” said I. “I am not accustomed to be thus questioned,” said he. “You look like a reasonable man—tell me, then, what are you waiting for?” “You would not understand,” he replied; “and how could you help me, if I were to tell you?” “I should not only understand, but would do all that I could,” said I. He did not + ------ + [INFO] Prompt processing: 0.579 s + [INFO] Full generation: 4.690 s + $ python llama.py --num-tokens 500 llama-7B.mlx.npz tokenizer.model 'Call me Ishmael. Some years ago never mind how long precisely, having little or no money in my purse, and nothing of greater consequence in my mind, I happened to be walking down Gower Street in the afternoon, in the heavy rain, and I saw a few steps off, a man in rags, who sat upon his bundle and looked hard into the wet as if he were going to cry. I watched him attentively for some time, and could not but observe that, though a numerous crowd was hurrying up and down, nobody took the least notice of him. I stopped at last, at a little distance, as if I had been in doubt, and after looking on a few minutes, walked straight up to him. He slowly raised his eyes, and fixed them upon me for a moment, without speaking, and then resumed his place and posture as before. I stood looking at him for a while, feeling very much pain at heart, and then said to him, “What are you doing there?” Something like a smile passed over his face, as he said slowly, “I am waiting for someone; but it has been three quarters of an hour now, and he has not come.” “What is it you are waiting for?” said I. Still he made no immediate reply, but again put his face down upon his hands, and did not' + [INFO] Loading model from disk: 5.628 s + Press enter to start generation + ------ + take his eyes from the ground. “What is it you are waiting for?” said I. “I am not accustomed to be thus questioned,” said he. “You look like a reasonable man—tell me, then, what are you waiting for?” “You would not understand,” he replied; “and how could you help me, if I were to tell you?” “I should not only understand, but would do all that I could,” said I. He did not reply, but still went on looking at the ground, and took hold of his bundle with a nervous trembling. I waited some time, and then resumed. “It is of no use to say you would not understand, if I were to tell you,” said he. “I have not told you why I am waiting for him,” said I. “And I am sure I should not understand,” replied he. “I will tell you then,” said I, “and, perhaps, you would not be surprised.” “No matter,” said he, “I shall be surprised anyhow; so tell me why you are waiting for him.” “He is my friend,” said I. “Yes,” said he, with a slight smile, “I know.” “He has been kind to me,” said I, “and I am waiting for him. I want to see him, and could have waited as I am now, for a much longer time.” “He will not soon come,” said he. “Unless he sees you here, he will not know of your having waited, and he will be very unlikely to come.” “No matter,” said I, “I shall wait for him.” “This is a strange thing,” said he, still with the same amused smile. “How did you know,” said I, “that he was coming? How should you be waiting?” “That is my secret,” said he. “And you expect him?” “Yes,” said I. “Are you disappointed then, if he does not come?” “No,” said I, “it is his secret, not mine.” “If he comes,” said he, “do you mean to go straight away?” “Yes,” said I, “I cannot be happy if I do not go straight away after him.” “Did you know this place before?” asked he. “Yes,” said I. “Is there any shop to buy food here?” “ + ------ + [INFO] Prompt processing: 0.633 s + [INFO] Full generation: 21.475 s + +Scripts +------- + +.. admonition:: Download the code + + The full example code is available in `mlx-examples `_. + +.. code: `https://github.com/ml-explore/mlx-examples/tree/main/llama`_ + +.. [1] Su, J., Lu, Y., Pan, S., Murtadha, A., Wen, B. and Liu, Y., 2021. + Roformer: Enhanced transformer with rotary position embedding. arXiv + preprint arXiv:2104.09864. +.. [2] Zhang, B. and Sennrich, R., 2019. Root mean square layer normalization. + Advances in Neural Information Processing Systems, 32. +.. [3] Shazeer, N., 2020. Glu variants improve transformer. arXiv preprint + arXiv:2002.05202. diff --git a/docs/build/html/_sources/examples/mlp.rst.txt b/docs/build/html/_sources/examples/mlp.rst.txt new file mode 100644 index 000000000..5763eeba0 --- /dev/null +++ b/docs/build/html/_sources/examples/mlp.rst.txt @@ -0,0 +1,131 @@ +.. _mlp: + +Multi-Layer Perceptron +---------------------- + +In this example we'll learn to use ``mlx.nn`` by implementing a simple +multi-layer perceptron to classify MNIST. + +As a first step import the MLX packages we need: + +.. code-block:: python + + import mlx.core as mx + import mlx.nn as nn + import mlx.optimizers as optim + + import numpy as np + + +The model is defined as the ``MLP`` class which inherits from +:class:`mlx.nn.Module`. We follow the standard idiom to make a new module: + +1. Define an ``__init__`` where the parameters and/or submodules are setup. See + the :ref:`Module class docs` for more information on how + :class:`mlx.nn.Module` registers parameters. +2. Define a ``__call__`` where the computation is implemented. + +.. code-block:: python + + class MLP(nn.Module): + def __init__( + self, num_layers: int, input_dim: int, hidden_dim: int, output_dim: int + ): + super().__init__() + layer_sizes = [input_dim] + [hidden_dim] * num_layers + [output_dim] + self.layers = [ + nn.Linear(idim, odim) + for idim, odim in zip(layer_sizes[:-1], layer_sizes[1:]) + ] + + def __call__(self, x): + for l in self.layers[:-1]: + x = mx.maximum(l(x), 0.0) + return self.layers[-1](x) + + +We define the loss function which takes the mean of the per-example cross +entropy loss. The ``mlx.nn.losses`` sub-package has implementations of some +commonly used loss functions. + +.. code-block:: python + + def loss_fn(model, X, y): + return mx.mean(nn.losses.cross_entropy(model(X), y)) + +We also need a function to compute the accuracy of the model on the validation +set: + +.. code-block:: python + + def eval_fn(model, X, y): + return mx.mean(mx.argmax(model(X), axis=1) == y) + +Next, setup the problem parameters and load the data: + +.. code-block:: python + + num_layers = 2 + hidden_dim = 32 + num_classes = 10 + batch_size = 256 + num_epochs = 10 + learning_rate = 1e-1 + + # Load the data + import mnist + train_images, train_labels, test_images, test_labels = map( + mx.array, mnist.mnist() + ) + +Since we're using SGD, we need an iterator which shuffles and constructs +minibatches of examples in the training set: + +.. code-block:: python + + def batch_iterate(batch_size, X, y): + perm = mx.array(np.random.permutation(y.size)) + for s in range(0, y.size, batch_size): + ids = perm[s : s + batch_size] + yield X[ids], y[ids] + + +Finally, we put it all together by instantiating the model, the +:class:`mlx.optimizers.SGD` optimizer, and running the training loop: + +.. code-block:: python + + # Load the model + model = MLP(num_layers, train_images.shape[-1], hidden_dim, num_classes) + mx.eval(model.parameters()) + + # Get a function which gives the loss and gradient of the + # loss with respect to the model's trainable parameters + loss_and_grad_fn = nn.value_and_grad(model, loss_fn) + + # Instantiate the optimizer + optimizer = optim.SGD(learning_rate=learning_rate) + + for e in range(num_epochs): + for X, y in batch_iterate(batch_size, train_images, train_labels): + loss, grads = loss_and_grad_fn(model, X, y) + + # Update the optimizer state and model parameters + # in a single call + optimizer.update(model, grads) + + # Force a graph evaluation + mx.eval(model.parameters(), optimizer.state) + + accuracy = eval_fn(model, test_images, test_labels) + print(f"Epoch {e}: Test accuracy {accuracy.item():.3f}") + + +.. note:: + The :func:`mlx.nn.value_and_grad` function is a convenience function to get + the gradient of a loss with respect to the trainable parameters of a model. + This should not be confused with :func:`mlx.core.value_and_grad`. + +The model should train to a decent accuracy (about 95%) after just a few passes +over the training set. The `full example `_ +is available in the MLX GitHub repo. diff --git a/docs/build/html/_sources/index.rst.txt b/docs/build/html/_sources/index.rst.txt new file mode 100644 index 000000000..a5bf3e9a2 --- /dev/null +++ b/docs/build/html/_sources/index.rst.txt @@ -0,0 +1,49 @@ +MLX +=== + +.. toctree:: + :caption: Install + :maxdepth: 1 + + install + +.. toctree:: + :caption: Usage + :maxdepth: 1 + + quick_start + using_streams + +.. toctree:: + :caption: Examples + :maxdepth: 1 + + examples/linear_regression + examples/mlp + examples/llama-inference + +.. toctree:: + :caption: Further Reading + :maxdepth: 1 + + dev/extensions + +.. toctree:: + :caption: Python API Reference + :maxdepth: 1 + + python/array + python/devices_and_streams + python/ops + python/random + python/transforms + python/fft + python/nn + python/optimizers + python/tree_utils + +.. toctree:: + :caption: C++ API Reference + :maxdepth: 1 + + cpp/ops diff --git a/docs/build/html/_sources/install.rst.txt b/docs/build/html/_sources/install.rst.txt new file mode 100644 index 000000000..fd4f9becd --- /dev/null +++ b/docs/build/html/_sources/install.rst.txt @@ -0,0 +1,102 @@ +Build and Install +================= + +Install from PyPI +----------------- + +MLX is available at Apple's internal PyPI repository. All you have to do to use +MLX with your own Apple silicon computer is + +.. code-block:: shell + + pip install apple-mlx -i https://pypi.apple.com/simple + +Build from source +----------------- + +Build Requirements +^^^^^^^^^^^^^^^^^^ + +- A C++ compiler with C++17 support (e.g. Clang >= 5.0) +- `cmake `_ -- version 3.24 or later, and ``make`` + + +Python API +^^^^^^^^^^ + +To build and install the MLX python library from source, first, clone MLX from +`its GitHub repo `_: + +.. code-block:: shell + + git clone git@github.com:ml-explore/mlx.git mlx && cd mlx + +Make sure that you have `pybind11 `_ +installed. You can install ``pybind11`` with ``pip``, ``brew`` or ``conda`` as follows: + +.. code-block:: shell + + pip install "pybind11[global]" + conda install pybind11 + brew install pybind11 + +Then simply build and install it using pip: + +.. code-block:: shell + + env CMAKE_BUILD_PARALLEL_LEVEL="" pip install . + + +C++ API +^^^^^^^ + +Currently, MLX must be built and installed from source. + +Similarly to the python library, to build and install the MLX C++ library start +by cloning MLX from `its GitHub repo +`_: + +.. code-block:: shell + + git clone git@github.com:ml-explore/mlx.git mlx && cd mlx + +Create a build directory and run CMake and make: + +.. code-block:: shell + + mkdir -p build && cd build + cmake .. && make -j + +Run tests with: + +.. code-block:: shell + + make test + +Install with: + +.. code-block:: shell + + make install + +Note that the built ``mlx.metallib`` file should be either at the same +directory as the executable statically linked to ``libmlx.a`` or the +preprocessor constant ``METAL_PATH`` should be defined at build time and it +should point to the path to the built metal library. + +.. list-table:: Build Options + :widths: 25 8 + :header-rows: 1 + + * - Option + - Default + * - MLX_BUILD_TESTS + - ON + * - MLX_BUILD_EXAMPLES + - OFF + * - MLX_BUILD_BENCHMARKS + - OFF + * - MLX_BUILD_METAL + - ON + * - MLX_BUILD_PYTHON_BINDINGS + - OFF diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.Device.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.Device.rst.txt new file mode 100644 index 000000000..eafe9082b --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.Device.rst.txt @@ -0,0 +1,28 @@ +mlx.core.Device +=============== + +.. currentmodule:: mlx.core + +.. autoclass:: Device + + + .. automethod:: __init__ + + + .. rubric:: Methods + + .. autosummary:: + + ~Device.__init__ + + + + + + .. rubric:: Attributes + + .. autosummary:: + + ~Device.type + + \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.Dtype.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.Dtype.rst.txt new file mode 100644 index 000000000..2ca0ac412 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.Dtype.rst.txt @@ -0,0 +1,28 @@ +mlx.core.Dtype +============== + +.. currentmodule:: mlx.core + +.. autoclass:: Dtype + + + .. automethod:: __init__ + + + .. rubric:: Methods + + .. autosummary:: + + ~Dtype.__init__ + + + + + + .. rubric:: Attributes + + .. autosummary:: + + ~Dtype.size + + \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.Stream.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.Stream.rst.txt new file mode 100644 index 000000000..14a00e70b --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.Stream.rst.txt @@ -0,0 +1,28 @@ +mlx.core.Stream +=============== + +.. currentmodule:: mlx.core + +.. autoclass:: Stream + + + .. automethod:: __init__ + + + .. rubric:: Methods + + .. autosummary:: + + ~Stream.__init__ + + + + + + .. rubric:: Attributes + + .. autosummary:: + + ~Stream.device + + \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.abs.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.abs.rst.txt new file mode 100644 index 000000000..90eca6f80 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.abs.rst.txt @@ -0,0 +1,6 @@ +mlx.core.abs +============ + +.. currentmodule:: mlx.core + +.. autofunction:: abs \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.add.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.add.rst.txt new file mode 100644 index 000000000..291a4dcb6 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.add.rst.txt @@ -0,0 +1,6 @@ +mlx.core.add +============ + +.. currentmodule:: mlx.core + +.. autofunction:: add \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.all.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.all.rst.txt new file mode 100644 index 000000000..77a7722eb --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.all.rst.txt @@ -0,0 +1,6 @@ +mlx.core.all +============ + +.. currentmodule:: mlx.core + +.. autofunction:: all \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.allclose.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.allclose.rst.txt new file mode 100644 index 000000000..db268abac --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.allclose.rst.txt @@ -0,0 +1,6 @@ +mlx.core.allclose +================= + +.. currentmodule:: mlx.core + +.. autofunction:: allclose \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.any.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.any.rst.txt new file mode 100644 index 000000000..e7f8e2db1 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.any.rst.txt @@ -0,0 +1,6 @@ +mlx.core.any +============ + +.. currentmodule:: mlx.core + +.. autofunction:: any \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.arange.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.arange.rst.txt new file mode 100644 index 000000000..6521b8bc4 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.arange.rst.txt @@ -0,0 +1,6 @@ +mlx.core.arange +=============== + +.. currentmodule:: mlx.core + +.. autofunction:: arange \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.arccos.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.arccos.rst.txt new file mode 100644 index 000000000..5474960fe --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.arccos.rst.txt @@ -0,0 +1,6 @@ +mlx.core.arccos +=============== + +.. currentmodule:: mlx.core + +.. autofunction:: arccos \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.arccosh.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.arccosh.rst.txt new file mode 100644 index 000000000..1925cef6f --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.arccosh.rst.txt @@ -0,0 +1,6 @@ +mlx.core.arccosh +================ + +.. currentmodule:: mlx.core + +.. autofunction:: arccosh \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.arcsin.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.arcsin.rst.txt new file mode 100644 index 000000000..c78d675df --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.arcsin.rst.txt @@ -0,0 +1,6 @@ +mlx.core.arcsin +=============== + +.. currentmodule:: mlx.core + +.. autofunction:: arcsin \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.arcsinh.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.arcsinh.rst.txt new file mode 100644 index 000000000..dc25a7f61 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.arcsinh.rst.txt @@ -0,0 +1,6 @@ +mlx.core.arcsinh +================ + +.. currentmodule:: mlx.core + +.. autofunction:: arcsinh \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.arctan.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.arctan.rst.txt new file mode 100644 index 000000000..d132be2d4 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.arctan.rst.txt @@ -0,0 +1,6 @@ +mlx.core.arctan +=============== + +.. currentmodule:: mlx.core + +.. autofunction:: arctan \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.arctanh.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.arctanh.rst.txt new file mode 100644 index 000000000..174616e82 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.arctanh.rst.txt @@ -0,0 +1,6 @@ +mlx.core.arctanh +================ + +.. currentmodule:: mlx.core + +.. autofunction:: arctanh \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.argmax.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.argmax.rst.txt new file mode 100644 index 000000000..925d39c3a --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.argmax.rst.txt @@ -0,0 +1,6 @@ +mlx.core.argmax +=============== + +.. currentmodule:: mlx.core + +.. autofunction:: argmax \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.argmin.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.argmin.rst.txt new file mode 100644 index 000000000..0f0696b9f --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.argmin.rst.txt @@ -0,0 +1,6 @@ +mlx.core.argmin +=============== + +.. currentmodule:: mlx.core + +.. autofunction:: argmin \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.argpartition.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.argpartition.rst.txt new file mode 100644 index 000000000..51ac000b9 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.argpartition.rst.txt @@ -0,0 +1,6 @@ +mlx.core.argpartition +===================== + +.. currentmodule:: mlx.core + +.. autofunction:: argpartition \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.argsort.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.argsort.rst.txt new file mode 100644 index 000000000..0353eeae4 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.argsort.rst.txt @@ -0,0 +1,6 @@ +mlx.core.argsort +================ + +.. currentmodule:: mlx.core + +.. autofunction:: argsort \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.T.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.T.rst.txt new file mode 100644 index 000000000..4008a16cc --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.T.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.T +================ + +.. currentmodule:: mlx.core + +.. autoproperty:: array.T \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.abs.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.abs.rst.txt new file mode 100644 index 000000000..9374238e7 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.abs.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.abs +================== + +.. currentmodule:: mlx.core + +.. automethod:: array.abs \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.all.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.all.rst.txt new file mode 100644 index 000000000..8ca4f0a4a --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.all.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.all +================== + +.. currentmodule:: mlx.core + +.. automethod:: array.all \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.any.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.any.rst.txt new file mode 100644 index 000000000..c24700a61 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.any.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.any +================== + +.. currentmodule:: mlx.core + +.. automethod:: array.any \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.argmax.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.argmax.rst.txt new file mode 100644 index 000000000..4e0aa26a6 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.argmax.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.argmax +===================== + +.. currentmodule:: mlx.core + +.. automethod:: array.argmax \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.argmin.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.argmin.rst.txt new file mode 100644 index 000000000..aa2061e72 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.argmin.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.argmin +===================== + +.. currentmodule:: mlx.core + +.. automethod:: array.argmin \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.astype.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.astype.rst.txt new file mode 100644 index 000000000..9370e618a --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.astype.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.astype +===================== + +.. currentmodule:: mlx.core + +.. automethod:: array.astype \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.cos.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.cos.rst.txt new file mode 100644 index 000000000..dec25f11d --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.cos.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.cos +================== + +.. currentmodule:: mlx.core + +.. automethod:: array.cos \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.dtype.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.dtype.rst.txt new file mode 100644 index 000000000..ad99ac69a --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.dtype.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.dtype +==================== + +.. currentmodule:: mlx.core + +.. autoproperty:: array.dtype \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.exp.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.exp.rst.txt new file mode 100644 index 000000000..e837cd4ad --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.exp.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.exp +================== + +.. currentmodule:: mlx.core + +.. automethod:: array.exp \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.item.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.item.rst.txt new file mode 100644 index 000000000..a592384c9 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.item.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.item +=================== + +.. currentmodule:: mlx.core + +.. automethod:: array.item \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.log.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.log.rst.txt new file mode 100644 index 000000000..cb28c3022 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.log.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.log +================== + +.. currentmodule:: mlx.core + +.. automethod:: array.log \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.log1p.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.log1p.rst.txt new file mode 100644 index 000000000..315678f81 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.log1p.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.log1p +==================== + +.. currentmodule:: mlx.core + +.. automethod:: array.log1p \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.logsumexp.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.logsumexp.rst.txt new file mode 100644 index 000000000..31e060adc --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.logsumexp.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.logsumexp +======================== + +.. currentmodule:: mlx.core + +.. automethod:: array.logsumexp \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.max.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.max.rst.txt new file mode 100644 index 000000000..52cc1312c --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.max.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.max +================== + +.. currentmodule:: mlx.core + +.. automethod:: array.max \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.mean.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.mean.rst.txt new file mode 100644 index 000000000..b871a0abb --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.mean.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.mean +=================== + +.. currentmodule:: mlx.core + +.. automethod:: array.mean \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.min.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.min.rst.txt new file mode 100644 index 000000000..ae971e61a --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.min.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.min +================== + +.. currentmodule:: mlx.core + +.. automethod:: array.min \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.ndim.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.ndim.rst.txt new file mode 100644 index 000000000..f351249cd --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.ndim.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.ndim +=================== + +.. currentmodule:: mlx.core + +.. autoproperty:: array.ndim \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.prod.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.prod.rst.txt new file mode 100644 index 000000000..b0779ebdd --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.prod.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.prod +=================== + +.. currentmodule:: mlx.core + +.. automethod:: array.prod \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.reciprocal.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.reciprocal.rst.txt new file mode 100644 index 000000000..c91a799fd --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.reciprocal.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.reciprocal +========================= + +.. currentmodule:: mlx.core + +.. automethod:: array.reciprocal \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.reshape.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.reshape.rst.txt new file mode 100644 index 000000000..7925714c2 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.reshape.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.reshape +====================== + +.. currentmodule:: mlx.core + +.. automethod:: array.reshape \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.rsqrt.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.rsqrt.rst.txt new file mode 100644 index 000000000..5c35f68a5 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.rsqrt.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.rsqrt +==================== + +.. currentmodule:: mlx.core + +.. automethod:: array.rsqrt \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.rst.txt new file mode 100644 index 000000000..21e66b5e4 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.rst.txt @@ -0,0 +1,66 @@ +mlx.core.array +============== + +.. currentmodule:: mlx.core + +.. autoclass:: array + + + .. automethod:: __init__ + + + .. rubric:: Methods + + .. autosummary:: + + ~array.__init__ + ~array.abs + ~array.all + ~array.any + ~array.argmax + ~array.argmin + ~array.astype + ~array.cos + ~array.cummax + ~array.cummin + ~array.cumprod + ~array.cumsum + ~array.exp + ~array.item + ~array.log + ~array.log10 + ~array.log1p + ~array.log2 + ~array.logsumexp + ~array.max + ~array.mean + ~array.min + ~array.prod + ~array.reciprocal + ~array.reshape + ~array.rsqrt + ~array.sin + ~array.split + ~array.sqrt + ~array.square + ~array.squeeze + ~array.sum + ~array.tolist + ~array.transpose + ~array.var + + + + + + .. rubric:: Attributes + + .. autosummary:: + + ~array.T + ~array.dtype + ~array.ndim + ~array.shape + ~array.size + + \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.shape.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.shape.rst.txt new file mode 100644 index 000000000..56eeb06bc --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.shape.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.shape +==================== + +.. currentmodule:: mlx.core + +.. autoproperty:: array.shape \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.sin.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.sin.rst.txt new file mode 100644 index 000000000..1b6810c8b --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.sin.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.sin +================== + +.. currentmodule:: mlx.core + +.. automethod:: array.sin \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.size.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.size.rst.txt new file mode 100644 index 000000000..8409fe262 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.size.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.size +=================== + +.. currentmodule:: mlx.core + +.. autoproperty:: array.size \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.split.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.split.rst.txt new file mode 100644 index 000000000..76fd5ac8c --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.split.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.split +==================== + +.. currentmodule:: mlx.core + +.. automethod:: array.split \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.sqrt.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.sqrt.rst.txt new file mode 100644 index 000000000..636867649 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.sqrt.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.sqrt +=================== + +.. currentmodule:: mlx.core + +.. automethod:: array.sqrt \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.square.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.square.rst.txt new file mode 100644 index 000000000..d543174c0 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.square.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.square +===================== + +.. currentmodule:: mlx.core + +.. automethod:: array.square \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.sum.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.sum.rst.txt new file mode 100644 index 000000000..456efc544 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.sum.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.sum +================== + +.. currentmodule:: mlx.core + +.. automethod:: array.sum \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.tolist.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.tolist.rst.txt new file mode 100644 index 000000000..0d8cdfaa0 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.tolist.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.tolist +===================== + +.. currentmodule:: mlx.core + +.. automethod:: array.tolist \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.transpose.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.transpose.rst.txt new file mode 100644 index 000000000..1c3845feb --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.transpose.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.transpose +======================== + +.. currentmodule:: mlx.core + +.. automethod:: array.transpose \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array.var.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array.var.rst.txt new file mode 100644 index 000000000..6d43ec8d6 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array.var.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array.var +================== + +.. currentmodule:: mlx.core + +.. automethod:: array.var \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.array_equal.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.array_equal.rst.txt new file mode 100644 index 000000000..09d8e25bb --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.array_equal.rst.txt @@ -0,0 +1,6 @@ +mlx.core.array\_equal +===================== + +.. currentmodule:: mlx.core + +.. autofunction:: array_equal \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.broadcast_to.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.broadcast_to.rst.txt new file mode 100644 index 000000000..a938ea97d --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.broadcast_to.rst.txt @@ -0,0 +1,6 @@ +mlx.core.broadcast\_to +====================== + +.. currentmodule:: mlx.core + +.. autofunction:: broadcast_to \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.concatenate.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.concatenate.rst.txt new file mode 100644 index 000000000..4352200e8 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.concatenate.rst.txt @@ -0,0 +1,6 @@ +mlx.core.concatenate +==================== + +.. currentmodule:: mlx.core + +.. autofunction:: concatenate \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.conv1d.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.conv1d.rst.txt new file mode 100644 index 000000000..47c9d78df --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.conv1d.rst.txt @@ -0,0 +1,6 @@ +mlx.core.conv1d +=============== + +.. currentmodule:: mlx.core + +.. autofunction:: conv1d \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.conv2d.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.conv2d.rst.txt new file mode 100644 index 000000000..694185321 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.conv2d.rst.txt @@ -0,0 +1,6 @@ +mlx.core.conv2d +=============== + +.. currentmodule:: mlx.core + +.. autofunction:: conv2d \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.convolve.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.convolve.rst.txt new file mode 100644 index 000000000..975dc71b3 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.convolve.rst.txt @@ -0,0 +1,6 @@ +mlx.core.convolve +================= + +.. currentmodule:: mlx.core + +.. autofunction:: convolve \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.cos.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.cos.rst.txt new file mode 100644 index 000000000..a3fa49a78 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.cos.rst.txt @@ -0,0 +1,6 @@ +mlx.core.cos +============ + +.. currentmodule:: mlx.core + +.. autofunction:: cos \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.cosh.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.cosh.rst.txt new file mode 100644 index 000000000..39437cfb7 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.cosh.rst.txt @@ -0,0 +1,6 @@ +mlx.core.cosh +============= + +.. currentmodule:: mlx.core + +.. autofunction:: cosh \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.default_device.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.default_device.rst.txt new file mode 100644 index 000000000..21affc69d --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.default_device.rst.txt @@ -0,0 +1,6 @@ +mlx.core.default\_device +======================== + +.. currentmodule:: mlx.core + +.. autofunction:: default_device \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.default_stream.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.default_stream.rst.txt new file mode 100644 index 000000000..96b7d41ee --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.default_stream.rst.txt @@ -0,0 +1,6 @@ +mlx.core.default\_stream +======================== + +.. currentmodule:: mlx.core + +.. autofunction:: default_stream \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.divide.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.divide.rst.txt new file mode 100644 index 000000000..7fe60717e --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.divide.rst.txt @@ -0,0 +1,6 @@ +mlx.core.divide +=============== + +.. currentmodule:: mlx.core + +.. autofunction:: divide \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.equal.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.equal.rst.txt new file mode 100644 index 000000000..f61f1189e --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.equal.rst.txt @@ -0,0 +1,6 @@ +mlx.core.equal +============== + +.. currentmodule:: mlx.core + +.. autofunction:: equal \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.erf.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.erf.rst.txt new file mode 100644 index 000000000..6682cd23a --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.erf.rst.txt @@ -0,0 +1,6 @@ +mlx.core.erf +============ + +.. currentmodule:: mlx.core + +.. autofunction:: erf \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.erfinv.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.erfinv.rst.txt new file mode 100644 index 000000000..0dde3b21c --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.erfinv.rst.txt @@ -0,0 +1,6 @@ +mlx.core.erfinv +=============== + +.. currentmodule:: mlx.core + +.. autofunction:: erfinv \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.eval.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.eval.rst.txt new file mode 100644 index 000000000..5b2891980 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.eval.rst.txt @@ -0,0 +1,6 @@ +mlx.core.eval +============= + +.. currentmodule:: mlx.core + +.. autofunction:: eval \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.exp.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.exp.rst.txt new file mode 100644 index 000000000..a70ea5d15 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.exp.rst.txt @@ -0,0 +1,6 @@ +mlx.core.exp +============ + +.. currentmodule:: mlx.core + +.. autofunction:: exp \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.expand_dims.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.expand_dims.rst.txt new file mode 100644 index 000000000..704825958 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.expand_dims.rst.txt @@ -0,0 +1,6 @@ +mlx.core.expand\_dims +===================== + +.. currentmodule:: mlx.core + +.. autofunction:: expand_dims \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.fft.fft.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.fft.rst.txt new file mode 100644 index 000000000..62c8eeb7f --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.fft.rst.txt @@ -0,0 +1,6 @@ +mlx.core.fft.fft +================ + +.. currentmodule:: mlx.core.fft + +.. autofunction:: fft \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.fft.fft2.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.fft2.rst.txt new file mode 100644 index 000000000..0f690a7d6 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.fft2.rst.txt @@ -0,0 +1,6 @@ +mlx.core.fft.fft2 +================= + +.. currentmodule:: mlx.core.fft + +.. autofunction:: fft2 \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.fft.fftn.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.fftn.rst.txt new file mode 100644 index 000000000..06845adfa --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.fftn.rst.txt @@ -0,0 +1,6 @@ +mlx.core.fft.fftn +================= + +.. currentmodule:: mlx.core.fft + +.. autofunction:: fftn \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.fft.ifft.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.ifft.rst.txt new file mode 100644 index 000000000..2b9d7f0e4 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.ifft.rst.txt @@ -0,0 +1,6 @@ +mlx.core.fft.ifft +================= + +.. currentmodule:: mlx.core.fft + +.. autofunction:: ifft \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.fft.ifft2.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.ifft2.rst.txt new file mode 100644 index 000000000..e07f9a409 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.ifft2.rst.txt @@ -0,0 +1,6 @@ +mlx.core.fft.ifft2 +================== + +.. currentmodule:: mlx.core.fft + +.. autofunction:: ifft2 \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.fft.ifftn.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.ifftn.rst.txt new file mode 100644 index 000000000..1e9b09941 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.ifftn.rst.txt @@ -0,0 +1,6 @@ +mlx.core.fft.ifftn +================== + +.. currentmodule:: mlx.core.fft + +.. autofunction:: ifftn \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.fft.irfft.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.irfft.rst.txt new file mode 100644 index 000000000..0a028a59e --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.irfft.rst.txt @@ -0,0 +1,6 @@ +mlx.core.fft.irfft +================== + +.. currentmodule:: mlx.core.fft + +.. autofunction:: irfft \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.fft.irfft2.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.irfft2.rst.txt new file mode 100644 index 000000000..b638433be --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.irfft2.rst.txt @@ -0,0 +1,6 @@ +mlx.core.fft.irfft2 +=================== + +.. currentmodule:: mlx.core.fft + +.. autofunction:: irfft2 \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.fft.irfftn.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.irfftn.rst.txt new file mode 100644 index 000000000..6668b2903 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.irfftn.rst.txt @@ -0,0 +1,6 @@ +mlx.core.fft.irfftn +=================== + +.. currentmodule:: mlx.core.fft + +.. autofunction:: irfftn \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.fft.rfft.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.rfft.rst.txt new file mode 100644 index 000000000..a9ab1056d --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.rfft.rst.txt @@ -0,0 +1,6 @@ +mlx.core.fft.rfft +================= + +.. currentmodule:: mlx.core.fft + +.. autofunction:: rfft \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.fft.rfft2.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.rfft2.rst.txt new file mode 100644 index 000000000..aef04184e --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.rfft2.rst.txt @@ -0,0 +1,6 @@ +mlx.core.fft.rfft2 +================== + +.. currentmodule:: mlx.core.fft + +.. autofunction:: rfft2 \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.fft.rfftn.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.rfftn.rst.txt new file mode 100644 index 000000000..95b852f74 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.fft.rfftn.rst.txt @@ -0,0 +1,6 @@ +mlx.core.fft.rfftn +================== + +.. currentmodule:: mlx.core.fft + +.. autofunction:: rfftn \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.full.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.full.rst.txt new file mode 100644 index 000000000..b4bdd26ea --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.full.rst.txt @@ -0,0 +1,6 @@ +mlx.core.full +============= + +.. currentmodule:: mlx.core + +.. autofunction:: full \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.grad.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.grad.rst.txt new file mode 100644 index 000000000..41e8dadea --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.grad.rst.txt @@ -0,0 +1,6 @@ +mlx.core.grad +============= + +.. currentmodule:: mlx.core + +.. autofunction:: grad \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.greater.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.greater.rst.txt new file mode 100644 index 000000000..db867b536 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.greater.rst.txt @@ -0,0 +1,6 @@ +mlx.core.greater +================ + +.. currentmodule:: mlx.core + +.. autofunction:: greater \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.greater_equal.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.greater_equal.rst.txt new file mode 100644 index 000000000..b6b1b0989 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.greater_equal.rst.txt @@ -0,0 +1,6 @@ +mlx.core.greater\_equal +======================= + +.. currentmodule:: mlx.core + +.. autofunction:: greater_equal \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.jvp.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.jvp.rst.txt new file mode 100644 index 000000000..b9f964c8b --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.jvp.rst.txt @@ -0,0 +1,6 @@ +mlx.core.jvp +============ + +.. currentmodule:: mlx.core + +.. autofunction:: jvp \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.less.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.less.rst.txt new file mode 100644 index 000000000..a5cf7dd43 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.less.rst.txt @@ -0,0 +1,6 @@ +mlx.core.less +============= + +.. currentmodule:: mlx.core + +.. autofunction:: less \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.less_equal.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.less_equal.rst.txt new file mode 100644 index 000000000..4398665a7 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.less_equal.rst.txt @@ -0,0 +1,6 @@ +mlx.core.less\_equal +==================== + +.. currentmodule:: mlx.core + +.. autofunction:: less_equal \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.load.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.load.rst.txt new file mode 100644 index 000000000..d02413969 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.load.rst.txt @@ -0,0 +1,6 @@ +mlx.core.load +============= + +.. currentmodule:: mlx.core + +.. autofunction:: load \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.log.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.log.rst.txt new file mode 100644 index 000000000..21fac68bb --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.log.rst.txt @@ -0,0 +1,6 @@ +mlx.core.log +============ + +.. currentmodule:: mlx.core + +.. autofunction:: log \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.log10.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.log10.rst.txt new file mode 100644 index 000000000..d22e63ac1 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.log10.rst.txt @@ -0,0 +1,6 @@ +mlx.core.log10 +============== + +.. currentmodule:: mlx.core + +.. autofunction:: log10 \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.log1p.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.log1p.rst.txt new file mode 100644 index 000000000..1331c99f1 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.log1p.rst.txt @@ -0,0 +1,6 @@ +mlx.core.log1p +============== + +.. currentmodule:: mlx.core + +.. autofunction:: log1p \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.log2.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.log2.rst.txt new file mode 100644 index 000000000..4c36e519c --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.log2.rst.txt @@ -0,0 +1,6 @@ +mlx.core.log2 +============= + +.. currentmodule:: mlx.core + +.. autofunction:: log2 \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.logaddexp.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.logaddexp.rst.txt new file mode 100644 index 000000000..89c747529 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.logaddexp.rst.txt @@ -0,0 +1,6 @@ +mlx.core.logaddexp +================== + +.. currentmodule:: mlx.core + +.. autofunction:: logaddexp \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.logical_not.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.logical_not.rst.txt new file mode 100644 index 000000000..7a83ea811 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.logical_not.rst.txt @@ -0,0 +1,6 @@ +mlx.core.logical\_not +===================== + +.. currentmodule:: mlx.core + +.. autofunction:: logical_not \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.logsumexp.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.logsumexp.rst.txt new file mode 100644 index 000000000..89f6b86e0 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.logsumexp.rst.txt @@ -0,0 +1,6 @@ +mlx.core.logsumexp +================== + +.. currentmodule:: mlx.core + +.. autofunction:: logsumexp \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.matmul.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.matmul.rst.txt new file mode 100644 index 000000000..261ae7fb2 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.matmul.rst.txt @@ -0,0 +1,6 @@ +mlx.core.matmul +=============== + +.. currentmodule:: mlx.core + +.. autofunction:: matmul \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.max.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.max.rst.txt new file mode 100644 index 000000000..74c2225e2 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.max.rst.txt @@ -0,0 +1,6 @@ +mlx.core.max +============ + +.. currentmodule:: mlx.core + +.. autofunction:: max \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.maximum.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.maximum.rst.txt new file mode 100644 index 000000000..82485a103 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.maximum.rst.txt @@ -0,0 +1,6 @@ +mlx.core.maximum +================ + +.. currentmodule:: mlx.core + +.. autofunction:: maximum \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.mean.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.mean.rst.txt new file mode 100644 index 000000000..1b6bd346d --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.mean.rst.txt @@ -0,0 +1,6 @@ +mlx.core.mean +============= + +.. currentmodule:: mlx.core + +.. autofunction:: mean \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.min.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.min.rst.txt new file mode 100644 index 000000000..4d6350e89 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.min.rst.txt @@ -0,0 +1,6 @@ +mlx.core.min +============ + +.. currentmodule:: mlx.core + +.. autofunction:: min \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.minimum.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.minimum.rst.txt new file mode 100644 index 000000000..c535a3e6e --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.minimum.rst.txt @@ -0,0 +1,6 @@ +mlx.core.minimum +================ + +.. currentmodule:: mlx.core + +.. autofunction:: minimum \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.multiply.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.multiply.rst.txt new file mode 100644 index 000000000..93b185dc6 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.multiply.rst.txt @@ -0,0 +1,6 @@ +mlx.core.multiply +================= + +.. currentmodule:: mlx.core + +.. autofunction:: multiply \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.negative.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.negative.rst.txt new file mode 100644 index 000000000..91a2cfbe0 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.negative.rst.txt @@ -0,0 +1,6 @@ +mlx.core.negative +================= + +.. currentmodule:: mlx.core + +.. autofunction:: negative \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.new_stream.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.new_stream.rst.txt new file mode 100644 index 000000000..f8aa5249b --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.new_stream.rst.txt @@ -0,0 +1,6 @@ +mlx.core.new\_stream +==================== + +.. currentmodule:: mlx.core + +.. autofunction:: new_stream \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.ones.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.ones.rst.txt new file mode 100644 index 000000000..d23dd1037 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.ones.rst.txt @@ -0,0 +1,6 @@ +mlx.core.ones +============= + +.. currentmodule:: mlx.core + +.. autofunction:: ones \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.ones_like.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.ones_like.rst.txt new file mode 100644 index 000000000..6a1b99feb --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.ones_like.rst.txt @@ -0,0 +1,6 @@ +mlx.core.ones\_like +=================== + +.. currentmodule:: mlx.core + +.. autofunction:: ones_like \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.pad.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.pad.rst.txt new file mode 100644 index 000000000..e2b9becb6 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.pad.rst.txt @@ -0,0 +1,6 @@ +mlx.core.pad +============ + +.. currentmodule:: mlx.core + +.. autofunction:: pad \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.partition.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.partition.rst.txt new file mode 100644 index 000000000..3a6f78fc7 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.partition.rst.txt @@ -0,0 +1,6 @@ +mlx.core.partition +================== + +.. currentmodule:: mlx.core + +.. autofunction:: partition \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.prod.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.prod.rst.txt new file mode 100644 index 000000000..e64cd3098 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.prod.rst.txt @@ -0,0 +1,6 @@ +mlx.core.prod +============= + +.. currentmodule:: mlx.core + +.. autofunction:: prod \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.random.bernoulli.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.random.bernoulli.rst.txt new file mode 100644 index 000000000..6dd4a2bae --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.random.bernoulli.rst.txt @@ -0,0 +1,6 @@ +mlx.core.random.bernoulli +========================= + +.. currentmodule:: mlx.core.random + +.. autofunction:: bernoulli \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.random.categorical.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.random.categorical.rst.txt new file mode 100644 index 000000000..b3533743b --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.random.categorical.rst.txt @@ -0,0 +1,6 @@ +mlx.core.random.categorical +=========================== + +.. currentmodule:: mlx.core.random + +.. autofunction:: categorical \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.random.gumbel.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.random.gumbel.rst.txt new file mode 100644 index 000000000..d01901fa0 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.random.gumbel.rst.txt @@ -0,0 +1,6 @@ +mlx.core.random.gumbel +====================== + +.. currentmodule:: mlx.core.random + +.. autofunction:: gumbel \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.random.key.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.random.key.rst.txt new file mode 100644 index 000000000..0939c55e0 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.random.key.rst.txt @@ -0,0 +1,6 @@ +mlx.core.random.key +=================== + +.. currentmodule:: mlx.core.random + +.. autofunction:: key \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.random.normal.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.random.normal.rst.txt new file mode 100644 index 000000000..ae23c986d --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.random.normal.rst.txt @@ -0,0 +1,6 @@ +mlx.core.random.normal +====================== + +.. currentmodule:: mlx.core.random + +.. autofunction:: normal \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.random.randint.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.random.randint.rst.txt new file mode 100644 index 000000000..957a85b84 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.random.randint.rst.txt @@ -0,0 +1,6 @@ +mlx.core.random.randint +======================= + +.. currentmodule:: mlx.core.random + +.. autofunction:: randint \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.random.seed.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.random.seed.rst.txt new file mode 100644 index 000000000..735f72e8a --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.random.seed.rst.txt @@ -0,0 +1,6 @@ +mlx.core.random.seed +==================== + +.. currentmodule:: mlx.core.random + +.. autofunction:: seed \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.random.split.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.random.split.rst.txt new file mode 100644 index 000000000..867611d14 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.random.split.rst.txt @@ -0,0 +1,6 @@ +mlx.core.random.split +===================== + +.. currentmodule:: mlx.core.random + +.. autofunction:: split \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.random.truncated_normal.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.random.truncated_normal.rst.txt new file mode 100644 index 000000000..c3cc7f4a6 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.random.truncated_normal.rst.txt @@ -0,0 +1,6 @@ +mlx.core.random.truncated\_normal +================================= + +.. currentmodule:: mlx.core.random + +.. autofunction:: truncated_normal \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.random.uniform.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.random.uniform.rst.txt new file mode 100644 index 000000000..9301d1d20 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.random.uniform.rst.txt @@ -0,0 +1,6 @@ +mlx.core.random.uniform +======================= + +.. currentmodule:: mlx.core.random + +.. autofunction:: uniform \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.reciprocal.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.reciprocal.rst.txt new file mode 100644 index 000000000..0f6890917 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.reciprocal.rst.txt @@ -0,0 +1,6 @@ +mlx.core.reciprocal +=================== + +.. currentmodule:: mlx.core + +.. autofunction:: reciprocal \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.reshape.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.reshape.rst.txt new file mode 100644 index 000000000..a7c5c9694 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.reshape.rst.txt @@ -0,0 +1,6 @@ +mlx.core.reshape +================ + +.. currentmodule:: mlx.core + +.. autofunction:: reshape \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.rsqrt.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.rsqrt.rst.txt new file mode 100644 index 000000000..0fa61dc40 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.rsqrt.rst.txt @@ -0,0 +1,6 @@ +mlx.core.rsqrt +============== + +.. currentmodule:: mlx.core + +.. autofunction:: rsqrt \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.save.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.save.rst.txt new file mode 100644 index 000000000..0e1a818b9 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.save.rst.txt @@ -0,0 +1,6 @@ +mlx.core.save +============= + +.. currentmodule:: mlx.core + +.. autofunction:: save \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.savez.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.savez.rst.txt new file mode 100644 index 000000000..80430b269 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.savez.rst.txt @@ -0,0 +1,6 @@ +mlx.core.savez +============== + +.. currentmodule:: mlx.core + +.. autofunction:: savez \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.savez_compressed.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.savez_compressed.rst.txt new file mode 100644 index 000000000..bc3a4e2fd --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.savez_compressed.rst.txt @@ -0,0 +1,6 @@ +mlx.core.savez\_compressed +========================== + +.. currentmodule:: mlx.core + +.. autofunction:: savez_compressed \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.set_default_device.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.set_default_device.rst.txt new file mode 100644 index 000000000..e381fcb7e --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.set_default_device.rst.txt @@ -0,0 +1,6 @@ +mlx.core.set\_default\_device +============================= + +.. currentmodule:: mlx.core + +.. autofunction:: set_default_device \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.set_default_stream.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.set_default_stream.rst.txt new file mode 100644 index 000000000..fd614926b --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.set_default_stream.rst.txt @@ -0,0 +1,6 @@ +mlx.core.set\_default\_stream +============================= + +.. currentmodule:: mlx.core + +.. autofunction:: set_default_stream \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.sigmoid.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.sigmoid.rst.txt new file mode 100644 index 000000000..69c020fdd --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.sigmoid.rst.txt @@ -0,0 +1,6 @@ +mlx.core.sigmoid +================ + +.. currentmodule:: mlx.core + +.. autofunction:: sigmoid \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.sign.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.sign.rst.txt new file mode 100644 index 000000000..19bf01602 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.sign.rst.txt @@ -0,0 +1,6 @@ +mlx.core.sign +============= + +.. currentmodule:: mlx.core + +.. autofunction:: sign \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.sin.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.sin.rst.txt new file mode 100644 index 000000000..edd0a0d25 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.sin.rst.txt @@ -0,0 +1,6 @@ +mlx.core.sin +============ + +.. currentmodule:: mlx.core + +.. autofunction:: sin \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.sinh.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.sinh.rst.txt new file mode 100644 index 000000000..59889278e --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.sinh.rst.txt @@ -0,0 +1,6 @@ +mlx.core.sinh +============= + +.. currentmodule:: mlx.core + +.. autofunction:: sinh \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.softmax.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.softmax.rst.txt new file mode 100644 index 000000000..241aeec7e --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.softmax.rst.txt @@ -0,0 +1,6 @@ +mlx.core.softmax +================ + +.. currentmodule:: mlx.core + +.. autofunction:: softmax \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.sort.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.sort.rst.txt new file mode 100644 index 000000000..d72b031a7 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.sort.rst.txt @@ -0,0 +1,6 @@ +mlx.core.sort +============= + +.. currentmodule:: mlx.core + +.. autofunction:: sort \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.split.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.split.rst.txt new file mode 100644 index 000000000..80d545f03 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.split.rst.txt @@ -0,0 +1,6 @@ +mlx.core.split +============== + +.. currentmodule:: mlx.core + +.. autofunction:: split \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.sqrt.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.sqrt.rst.txt new file mode 100644 index 000000000..5020c386f --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.sqrt.rst.txt @@ -0,0 +1,6 @@ +mlx.core.sqrt +============= + +.. currentmodule:: mlx.core + +.. autofunction:: sqrt \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.square.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.square.rst.txt new file mode 100644 index 000000000..06c6277b9 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.square.rst.txt @@ -0,0 +1,6 @@ +mlx.core.square +=============== + +.. currentmodule:: mlx.core + +.. autofunction:: square \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.squeeze.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.squeeze.rst.txt new file mode 100644 index 000000000..7c4b7ae76 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.squeeze.rst.txt @@ -0,0 +1,6 @@ +mlx.core.squeeze +================ + +.. currentmodule:: mlx.core + +.. autofunction:: squeeze \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.stop_gradient.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.stop_gradient.rst.txt new file mode 100644 index 000000000..309f02375 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.stop_gradient.rst.txt @@ -0,0 +1,6 @@ +mlx.core.stop\_gradient +======================= + +.. currentmodule:: mlx.core + +.. autofunction:: stop_gradient \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.subtract.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.subtract.rst.txt new file mode 100644 index 000000000..b25e6c50b --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.subtract.rst.txt @@ -0,0 +1,6 @@ +mlx.core.subtract +================= + +.. currentmodule:: mlx.core + +.. autofunction:: subtract \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.sum.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.sum.rst.txt new file mode 100644 index 000000000..6800c8c6b --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.sum.rst.txt @@ -0,0 +1,6 @@ +mlx.core.sum +============ + +.. currentmodule:: mlx.core + +.. autofunction:: sum \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.take.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.take.rst.txt new file mode 100644 index 000000000..ec6579a79 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.take.rst.txt @@ -0,0 +1,6 @@ +mlx.core.take +============= + +.. currentmodule:: mlx.core + +.. autofunction:: take \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.take_along_axis.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.take_along_axis.rst.txt new file mode 100644 index 000000000..3897da7dc --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.take_along_axis.rst.txt @@ -0,0 +1,6 @@ +mlx.core.take\_along\_axis +========================== + +.. currentmodule:: mlx.core + +.. autofunction:: take_along_axis \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.tan.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.tan.rst.txt new file mode 100644 index 000000000..d4aed157e --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.tan.rst.txt @@ -0,0 +1,6 @@ +mlx.core.tan +============ + +.. currentmodule:: mlx.core + +.. autofunction:: tan \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.tanh.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.tanh.rst.txt new file mode 100644 index 000000000..8251066c0 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.tanh.rst.txt @@ -0,0 +1,6 @@ +mlx.core.tanh +============= + +.. currentmodule:: mlx.core + +.. autofunction:: tanh \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.transpose.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.transpose.rst.txt new file mode 100644 index 000000000..2648d84a7 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.transpose.rst.txt @@ -0,0 +1,6 @@ +mlx.core.transpose +================== + +.. currentmodule:: mlx.core + +.. autofunction:: transpose \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.value_and_grad.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.value_and_grad.rst.txt new file mode 100644 index 000000000..b400a663d --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.value_and_grad.rst.txt @@ -0,0 +1,6 @@ +mlx.core.value\_and\_grad +========================= + +.. currentmodule:: mlx.core + +.. autofunction:: value_and_grad \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.var.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.var.rst.txt new file mode 100644 index 000000000..8217874af --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.var.rst.txt @@ -0,0 +1,6 @@ +mlx.core.var +============ + +.. currentmodule:: mlx.core + +.. autofunction:: var \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.vjp.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.vjp.rst.txt new file mode 100644 index 000000000..6dfc91397 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.vjp.rst.txt @@ -0,0 +1,6 @@ +mlx.core.vjp +============ + +.. currentmodule:: mlx.core + +.. autofunction:: vjp \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.vmap.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.vmap.rst.txt new file mode 100644 index 000000000..d937e6071 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.vmap.rst.txt @@ -0,0 +1,6 @@ +mlx.core.vmap +============= + +.. currentmodule:: mlx.core + +.. autofunction:: vmap \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.where.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.where.rst.txt new file mode 100644 index 000000000..fbaa4504a --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.where.rst.txt @@ -0,0 +1,6 @@ +mlx.core.where +============== + +.. currentmodule:: mlx.core + +.. autofunction:: where \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.zeros.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.zeros.rst.txt new file mode 100644 index 000000000..9738ac24b --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.zeros.rst.txt @@ -0,0 +1,6 @@ +mlx.core.zeros +============== + +.. currentmodule:: mlx.core + +.. autofunction:: zeros \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.core.zeros_like.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.core.zeros_like.rst.txt new file mode 100644 index 000000000..8de29dbe5 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.core.zeros_like.rst.txt @@ -0,0 +1,6 @@ +mlx.core.zeros\_like +==================== + +.. currentmodule:: mlx.core + +.. autofunction:: zeros_like \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.nn.Conv1d.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.nn.Conv1d.rst.txt new file mode 100644 index 000000000..c4128b83b --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.nn.Conv1d.rst.txt @@ -0,0 +1,8 @@ +mlx.nn.Conv1d +============= + +.. currentmodule:: mlx.nn + +.. autoclass:: Conv1d + + \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.nn.Conv2d.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.nn.Conv2d.rst.txt new file mode 100644 index 000000000..7bd1f08bb --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.nn.Conv2d.rst.txt @@ -0,0 +1,8 @@ +mlx.nn.Conv2d +============= + +.. currentmodule:: mlx.nn + +.. autoclass:: Conv2d + + \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.nn.Embedding.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.nn.Embedding.rst.txt new file mode 100644 index 000000000..ad2f3f2ce --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.nn.Embedding.rst.txt @@ -0,0 +1,8 @@ +mlx.nn.Embedding +================ + +.. currentmodule:: mlx.nn + +.. autoclass:: Embedding + + \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.nn.GELU.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.nn.GELU.rst.txt new file mode 100644 index 000000000..c963c84f2 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.nn.GELU.rst.txt @@ -0,0 +1,8 @@ +mlx.nn.GELU +=========== + +.. currentmodule:: mlx.nn + +.. autoclass:: GELU + + \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.nn.GroupNorm.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.nn.GroupNorm.rst.txt new file mode 100644 index 000000000..762d9ffea --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.nn.GroupNorm.rst.txt @@ -0,0 +1,8 @@ +mlx.nn.GroupNorm +================ + +.. currentmodule:: mlx.nn + +.. autoclass:: GroupNorm + + \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.nn.LayerNorm.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.nn.LayerNorm.rst.txt new file mode 100644 index 000000000..cc0ac117d --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.nn.LayerNorm.rst.txt @@ -0,0 +1,8 @@ +mlx.nn.LayerNorm +================ + +.. currentmodule:: mlx.nn + +.. autoclass:: LayerNorm + + \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.nn.Linear.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.nn.Linear.rst.txt new file mode 100644 index 000000000..627e6e6e6 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.nn.Linear.rst.txt @@ -0,0 +1,8 @@ +mlx.nn.Linear +============= + +.. currentmodule:: mlx.nn + +.. autoclass:: Linear + + \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.nn.MultiHeadAttention.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.nn.MultiHeadAttention.rst.txt new file mode 100644 index 000000000..2c3a8fcc1 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.nn.MultiHeadAttention.rst.txt @@ -0,0 +1,8 @@ +mlx.nn.MultiHeadAttention +========================= + +.. currentmodule:: mlx.nn + +.. autoclass:: MultiHeadAttention + + \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.nn.RMSNorm.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.nn.RMSNorm.rst.txt new file mode 100644 index 000000000..474b1355d --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.nn.RMSNorm.rst.txt @@ -0,0 +1,8 @@ +mlx.nn.RMSNorm +============== + +.. currentmodule:: mlx.nn + +.. autoclass:: RMSNorm + + \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.nn.ReLU.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.nn.ReLU.rst.txt new file mode 100644 index 000000000..944917de9 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.nn.ReLU.rst.txt @@ -0,0 +1,8 @@ +mlx.nn.ReLU +=========== + +.. currentmodule:: mlx.nn + +.. autoclass:: ReLU + + \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.nn.RoPE.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.nn.RoPE.rst.txt new file mode 100644 index 000000000..392fbab7b --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.nn.RoPE.rst.txt @@ -0,0 +1,8 @@ +mlx.nn.RoPE +=========== + +.. currentmodule:: mlx.nn + +.. autoclass:: RoPE + + \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.nn.Sequential.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.nn.Sequential.rst.txt new file mode 100644 index 000000000..af6ee04ab --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.nn.Sequential.rst.txt @@ -0,0 +1,8 @@ +mlx.nn.Sequential +================= + +.. currentmodule:: mlx.nn + +.. autoclass:: Sequential + + \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.nn.SiLU.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.nn.SiLU.rst.txt new file mode 100644 index 000000000..85069c9d5 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.nn.SiLU.rst.txt @@ -0,0 +1,8 @@ +mlx.nn.SiLU +=========== + +.. currentmodule:: mlx.nn + +.. autoclass:: SiLU + + \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.nn.value_and_grad.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.nn.value_and_grad.rst.txt new file mode 100644 index 000000000..103cb3606 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.nn.value_and_grad.rst.txt @@ -0,0 +1,6 @@ +mlx.nn.value\_and\_grad +======================= + +.. currentmodule:: mlx.nn + +.. autofunction:: value_and_grad \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.optimizers.Adam.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.optimizers.Adam.rst.txt new file mode 100644 index 000000000..074080ea6 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.optimizers.Adam.rst.txt @@ -0,0 +1,18 @@ +mlx.optimizers.Adam +=================== + +.. currentmodule:: mlx.optimizers + +.. autoclass:: Adam + + + + + .. rubric:: Methods + + .. autosummary:: + + ~Adam.__init__ + ~Adam.apply_single + + diff --git a/docs/build/html/_sources/python/_autosummary/mlx.optimizers.Optimizer.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.optimizers.Optimizer.rst.txt new file mode 100644 index 000000000..613eb02cf --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.optimizers.Optimizer.rst.txt @@ -0,0 +1,20 @@ +mlx.optimizers.Optimizer +======================== + +.. currentmodule:: mlx.optimizers + +.. autoclass:: Optimizer + + + + + .. rubric:: Methods + + .. autosummary:: + + ~Optimizer.__init__ + ~Optimizer.apply_gradients + ~Optimizer.apply_single + ~Optimizer.update + + diff --git a/docs/build/html/_sources/python/_autosummary/mlx.optimizers.OptimizerState.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.optimizers.OptimizerState.rst.txt new file mode 100644 index 000000000..b319b6d09 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.optimizers.OptimizerState.rst.txt @@ -0,0 +1,17 @@ +mlx.optimizers.OptimizerState +============================= + +.. currentmodule:: mlx.optimizers + +.. autoclass:: OptimizerState + + + + + .. rubric:: Methods + + .. autosummary:: + + ~OptimizerState.get + + diff --git a/docs/build/html/_sources/python/_autosummary/mlx.optimizers.SGD.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.optimizers.SGD.rst.txt new file mode 100644 index 000000000..35a9759ed --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.optimizers.SGD.rst.txt @@ -0,0 +1,18 @@ +mlx.optimizers.SGD +================== + +.. currentmodule:: mlx.optimizers + +.. autoclass:: SGD + + + + + .. rubric:: Methods + + .. autosummary:: + + ~SGD.__init__ + ~SGD.apply_single + + diff --git a/docs/build/html/_sources/python/_autosummary/mlx.utils.tree_flatten.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.utils.tree_flatten.rst.txt new file mode 100644 index 000000000..554cf3fb4 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.utils.tree_flatten.rst.txt @@ -0,0 +1,6 @@ +mlx.utils.tree\_flatten +======================= + +.. currentmodule:: mlx.utils + +.. autofunction:: tree_flatten \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.utils.tree_map.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.utils.tree_map.rst.txt new file mode 100644 index 000000000..0475ff44b --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.utils.tree_map.rst.txt @@ -0,0 +1,6 @@ +mlx.utils.tree\_map +=================== + +.. currentmodule:: mlx.utils + +.. autofunction:: tree_map \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary/mlx.utils.tree_unflatten.rst.txt b/docs/build/html/_sources/python/_autosummary/mlx.utils.tree_unflatten.rst.txt new file mode 100644 index 000000000..c8d6aa593 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary/mlx.utils.tree_unflatten.rst.txt @@ -0,0 +1,6 @@ +mlx.utils.tree\_unflatten +========================= + +.. currentmodule:: mlx.utils + +.. autofunction:: tree_unflatten \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary_functions/mlx.nn.gelu.rst.txt b/docs/build/html/_sources/python/_autosummary_functions/mlx.nn.gelu.rst.txt new file mode 100644 index 000000000..3e1668eb6 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary_functions/mlx.nn.gelu.rst.txt @@ -0,0 +1,8 @@ +mlx.nn.gelu +=========== + +.. currentmodule:: mlx.nn + +.. autoclass:: gelu + + \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary_functions/mlx.nn.gelu_approx.rst.txt b/docs/build/html/_sources/python/_autosummary_functions/mlx.nn.gelu_approx.rst.txt new file mode 100644 index 000000000..de08dc88c --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary_functions/mlx.nn.gelu_approx.rst.txt @@ -0,0 +1,8 @@ +mlx.nn.gelu\_approx +=================== + +.. currentmodule:: mlx.nn + +.. autoclass:: gelu_approx + + \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary_functions/mlx.nn.gelu_fast_approx.rst.txt b/docs/build/html/_sources/python/_autosummary_functions/mlx.nn.gelu_fast_approx.rst.txt new file mode 100644 index 000000000..c84114e6c --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary_functions/mlx.nn.gelu_fast_approx.rst.txt @@ -0,0 +1,8 @@ +mlx.nn.gelu\_fast\_approx +========================= + +.. currentmodule:: mlx.nn + +.. autoclass:: gelu_fast_approx + + \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary_functions/mlx.nn.relu.rst.txt b/docs/build/html/_sources/python/_autosummary_functions/mlx.nn.relu.rst.txt new file mode 100644 index 000000000..93a69272a --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary_functions/mlx.nn.relu.rst.txt @@ -0,0 +1,8 @@ +mlx.nn.relu +=========== + +.. currentmodule:: mlx.nn + +.. autoclass:: relu + + \ No newline at end of file diff --git a/docs/build/html/_sources/python/_autosummary_functions/mlx.nn.silu.rst.txt b/docs/build/html/_sources/python/_autosummary_functions/mlx.nn.silu.rst.txt new file mode 100644 index 000000000..b30c17b06 --- /dev/null +++ b/docs/build/html/_sources/python/_autosummary_functions/mlx.nn.silu.rst.txt @@ -0,0 +1,8 @@ +mlx.nn.silu +=========== + +.. currentmodule:: mlx.nn + +.. autoclass:: silu + + \ No newline at end of file diff --git a/docs/build/html/_sources/python/array.rst.txt b/docs/build/html/_sources/python/array.rst.txt new file mode 100644 index 000000000..96ddd32b3 --- /dev/null +++ b/docs/build/html/_sources/python/array.rst.txt @@ -0,0 +1,45 @@ +.. _array: + +Array +===== + +.. currentmodule:: mlx.core + +.. autosummary:: + :toctree: _autosummary + + array + array.astype + array.item + array.tolist + array.dtype + array.ndim + array.shape + array.size + Dtype + array.abs + array.all + array.any + array.argmax + array.argmin + array.cos + array.dtype + array.exp + array.log + array.log1p + array.logsumexp + array.max + array.mean + array.min + array.prod + array.reciprocal + array.reshape + array.rsqrt + array.sin + array.split + array.sqrt + array.square + array.sum + array.transpose + array.T + array.var diff --git a/docs/build/html/_sources/python/data_types.rst.txt b/docs/build/html/_sources/python/data_types.rst.txt new file mode 100644 index 000000000..cbb5c9a3f --- /dev/null +++ b/docs/build/html/_sources/python/data_types.rst.txt @@ -0,0 +1,52 @@ +.. _data_types: + +:orphan: + +Data Types +========== + +.. currentmodule:: mlx.core + +The default floating point type is ``float32`` and the default integer type is +``int32``. The table below shows supported values for :obj:`Dtype`. + +.. list-table:: Supported Data Types + :widths: 5 3 20 + :header-rows: 1 + + * - Type + - Bytes + - Description + * - ``bool_`` + - 1 + - Boolean (``True``, ``False``) data type + * - ``uint8`` + - 1 + - 8-bit unsigned integer + * - ``uint16`` + - 2 + - 16-bit unsigned integer + * - ``uint32`` + - 4 + - 32-bit unsigned integer + * - ``uint32`` + - 8 + - 32-bit unsigned integer + * - ``int8`` + - 1 + - 8-bit signed integer + * - ``int16`` + - 2 + - 16-bit signed integer + * - ``int32`` + - 4 + - 32-bit signed integer + * - ``int64`` + - 8 + - 64-bit signed integer + * - ``float16`` + - 2 + - 16-bit float, only available with `ARM C language extensions `_ + * - ``float32`` + - 4 + - 32-bit float diff --git a/docs/build/html/_sources/python/devices_and_streams.rst.txt b/docs/build/html/_sources/python/devices_and_streams.rst.txt new file mode 100644 index 000000000..bb9dfae2f --- /dev/null +++ b/docs/build/html/_sources/python/devices_and_streams.rst.txt @@ -0,0 +1,17 @@ +.. _devices_and_streams: + +Devices and Streams +=================== + +.. currentmodule:: mlx.core + +.. autosummary:: + :toctree: _autosummary + + Device + default_device + set_default_device + Stream + default_stream + new_stream + set_default_stream diff --git a/docs/build/html/_sources/python/fft.rst.txt b/docs/build/html/_sources/python/fft.rst.txt new file mode 100644 index 000000000..9e4be084b --- /dev/null +++ b/docs/build/html/_sources/python/fft.rst.txt @@ -0,0 +1,22 @@ +.. _fft: + +FFT +=== + +.. currentmodule:: mlx.core.fft + +.. autosummary:: + :toctree: _autosummary + + fft + ifft + fft2 + ifft2 + fftn + ifftn + rfft + irfft + rfft2 + irfft2 + rfftn + irfftn diff --git a/docs/build/html/_sources/python/nn.rst.txt b/docs/build/html/_sources/python/nn.rst.txt new file mode 100644 index 000000000..114fd8a90 --- /dev/null +++ b/docs/build/html/_sources/python/nn.rst.txt @@ -0,0 +1,172 @@ +.. _nn: + +.. currentmodule:: mlx.nn + +Neural Networks +=============== + +Writing arbitrarily complex neural networks in MLX can be done using only +:class:`mlx.core.array` and :meth:`mlx.core.value_and_grad`. However, this requires the +user to write again and again the same simple neural network operations as well +as handle all the parameter state and initialization manually and explicitly. + +The module :mod:`mlx.nn` solves this problem by providing an intuitive way of +composing neural network layers, initializing their parameters, freezing them +for finetuning and more. + +Quick Start with Neural Networks +--------------------------------- + +.. code-block:: python + + import mlx.core as mx + import mlx.nn as nn + + class MLP(nn.Module): + def __init__(self, in_dims: int, out_dims: int): + super().__init__() + + self.layers = [ + nn.Linear(in_dims, 128), + nn.Linear(128, 128), + nn.Linear(128, out_dims), + ] + + def __call__(self, x): + for i, l in enumerate(self.layers): + x = mx.maximum(x, 0) if i > 0 else x + x = l(x) + return x + + # The model is created with all its parameters but nothing is initialized + # yet because MLX is lazily evaluated + mlp = MLP(2, 10) + + # We can access its parameters by calling mlp.parameters() + params = mlp.parameters() + print(params["layers"][0]["weight"].shape) + + # Printing a parameter will cause it to be evaluated and thus initialized + print(params["layers"][0]) + + # We can also force evaluate all parameters to initialize the model + mx.eval(mlp.parameters()) + + # A simple loss function. + # NOTE: It doesn't matter how it uses the mlp model. It currently captures + # it from the local scope. It could be a positional argument or a + # keyword argument. + def l2_loss(x, y): + y_hat = mlp(x) + return (y_hat - y).square().mean() + + # Calling `nn.value_and_grad` instead of `mx.value_and_grad` returns the + # gradient with respect to `mlp.trainable_parameters()` + loss_and_grad = nn.value_and_grad(mlp, l2_loss) + + +.. _module_class: + +The Module Class +---------------- + +The workhorse of any neural network library is the :class:`Module` class. In +MLX the :class:`Module` class is a container of :class:`mlx.core.array` or +:class:`Module` instances. Its main function is to provide a way to +recursively **access** and **update** its parameters and those of its +submodules. + +Parameters +^^^^^^^^^^ + +A parameter of a module is any public member of type :class:`mlx.core.array` (its +name should not start with ``_``). It can be arbitrarily nested in other +:class:`Module` instances or lists and dictionaries. + +:meth:`Module.parameters` can be used to extract a nested dictionary with all +the parameters of a module and its submodules. + +A :class:`Module` can also keep track of "frozen" parameters. +:meth:`Module.trainable_parameters` returns only the subset of +:meth:`Module.parameters` that is not frozen. When using +:meth:`mlx.nn.value_and_grad` the gradients returned will be with respect to these +trainable parameters. + +Updating the parameters +^^^^^^^^^^^^^^^^^^^^^^^ + +MLX modules allow accessing and updating individual parameters. However, most +times we need to update large subsets of a module's parameters. This action is +performed by :meth:`Module.update`. + +Value and grad +-------------- + +Using a :class:`Module` does not preclude using MLX's high order function +transformations (:meth:`mlx.core.value_and_grad`, :meth:`mlx.core.grad`, etc.). However, +these function transformations assume pure functions, namely the parameters +should be passed as an argument to the function being transformed. + +There is an easy pattern to achieve that with MLX modules + +.. code-block:: python + + model = ... + + def f(params, other_inputs): + model.update(params) # <---- Necessary to make the model use the passed parameters + return model(other_inputs) + + f(model.trainable_parameters(), mx.zeros((10,))) + +However, :meth:`mlx.nn.value_and_grad` provides precisely this pattern and only +computes the gradients with respect to the trainable parameters of the model. + +In detail: + +- it wraps the passed function with a function that calls :meth:`Module.update` + to make sure the model is using the provided parameters. +- it calls :meth:`mlx.core.value_and_grad` to transform the function into a function + that also computes the gradients with respect to the passed parameters. +- it wraps the returned function with a function that passes the trainable + parameters as the first argument to the function returned by + :meth:`mlx.core.value_and_grad` + +.. autosummary:: + :toctree: _autosummary + + value_and_grad + +Neural Network Layers +--------------------- + +.. autosummary:: + :toctree: _autosummary + :template: nn-module-template.rst + + Embedding + ReLU + GELU + SiLU + Linear + Conv1d + Conv2d + LayerNorm + RMSNorm + GroupNorm + RoPE + MultiHeadAttention + Sequential + +Layers without parameters (e.g. activation functions) are also provided as +simple functions. + +.. autosummary:: + :toctree: _autosummary_functions + :template: nn-module-template.rst + + gelu + gelu_approx + gelu_fast_approx + relu + silu diff --git a/docs/build/html/_sources/python/nn/module.rst.txt b/docs/build/html/_sources/python/nn/module.rst.txt new file mode 100644 index 000000000..e14ba96f4 --- /dev/null +++ b/docs/build/html/_sources/python/nn/module.rst.txt @@ -0,0 +1,7 @@ +mlx.nn.Module +============= + +.. currentmodule:: mlx.nn + +.. autoclass:: Module + :members: diff --git a/docs/build/html/_sources/python/ops.rst.txt b/docs/build/html/_sources/python/ops.rst.txt new file mode 100644 index 000000000..450588536 --- /dev/null +++ b/docs/build/html/_sources/python/ops.rst.txt @@ -0,0 +1,94 @@ +.. _ops: + +Operations +========== + +.. currentmodule:: mlx.core + +.. autosummary:: + :toctree: _autosummary + + abs + add + all + allclose + any + arange + arccos + arccosh + arcsin + arcsinh + arctan + arctanh + argmax + argmin + argpartition + argsort + array_equal + broadcast_to + concatenate + convolve + conv1d + conv2d + cos + cosh + divide + equal + erf + erfinv + exp + expand_dims + full + greater + greater_equal + less + less_equal + load + log + log2 + log10 + log1p + logaddexp + logical_not + logsumexp + matmul + max + maximum + mean + min + minimum + multiply + negative + ones + ones_like + partition + pad + prod + reciprocal + reshape + rsqrt + save + savez + savez_compressed + sigmoid + sign + sin + sinh + softmax + sort + split + sqrt + square + squeeze + stop_gradient + subtract + sum + take + take_along_axis + tan + tanh + transpose + var + where + zeros + zeros_like diff --git a/docs/build/html/_sources/python/optimizers.rst.txt b/docs/build/html/_sources/python/optimizers.rst.txt new file mode 100644 index 000000000..7f5d3a067 --- /dev/null +++ b/docs/build/html/_sources/python/optimizers.rst.txt @@ -0,0 +1,41 @@ +.. _optimizers: + +Optimizers +========== + +The optimizers in MLX can be used both with :mod:`mlx.nn` but also with pure +:mod:`mlx.core` functions. A typical example involves calling +:meth:`Optimizer.update` to update a model's parameters based on the loss +gradients and subsequently calling :func:`mlx.core.eval` to evaluate both the +model's parameters and the **optimizer state**. + +.. code-block:: python + + # Create a model + model = MLP(num_layers, train_images.shape[-1], hidden_dim, num_classes) + mx.eval(model.parameters()) + + # Create the gradient function and the optimizer + loss_and_grad_fn = nn.value_and_grad(model, loss_fn) + optimizer = optim.SGD(learning_rate=learning_rate) + + for e in range(num_epochs): + for X, y in batch_iterate(batch_size, train_images, train_labels): + loss, grads = loss_and_grad_fn(model, X, y) + + # Update the model with the gradients. So far no computation has happened. + optimizer.update(model, grads) + + # Compute the new parameters but also the optimizer state. + mx.eval(model.parameters(), optimizer.state) + +.. currentmodule:: mlx.optimizers + +.. autosummary:: + :toctree: _autosummary + :template: optimizers-template.rst + + OptimizerState + Optimizer + SGD + Adam diff --git a/docs/build/html/_sources/python/random.rst.txt b/docs/build/html/_sources/python/random.rst.txt new file mode 100644 index 000000000..8ac3eaa11 --- /dev/null +++ b/docs/build/html/_sources/python/random.rst.txt @@ -0,0 +1,45 @@ +.. _random: + +Random +====== + +Random sampling functions in MLX use an implicit global PRNG state by default. +However, all function take an optional ``key`` keyword argument for when more +fine-grained control or explicit state management is needed. + +For example, you can generate random numbers with: + +.. code-block:: python + + for _ in range(3): + print(mx.random.uniform()) + +which will print a sequence of unique pseudo random numbers. Alternatively you +can explicitly set the key: + +.. code-block:: python + + key = mx.random.key(0) + for _ in range(3): + print(mx.random.uniform(key=key)) + +which will yield the same pseudo random number at each iteration. + +Following `JAX's PRNG design `_ +we use a splittable version of Threefry, which is a counter-based PRNG. + +.. currentmodule:: mlx.core.random + +.. autosummary:: + :toctree: _autosummary + + seed + key + split + bernoulli + categorical + gumbel + normal + randint + uniform + truncated_normal diff --git a/docs/build/html/_sources/python/transforms.rst.txt b/docs/build/html/_sources/python/transforms.rst.txt new file mode 100644 index 000000000..cc8d681d5 --- /dev/null +++ b/docs/build/html/_sources/python/transforms.rst.txt @@ -0,0 +1,16 @@ +.. _transforms: + +Transforms +========== + +.. currentmodule:: mlx.core + +.. autosummary:: + :toctree: _autosummary + + eval + grad + value_and_grad + jvp + vjp + vmap diff --git a/docs/build/html/_sources/python/tree_utils.rst.txt b/docs/build/html/_sources/python/tree_utils.rst.txt new file mode 100644 index 000000000..84d5afa9b --- /dev/null +++ b/docs/build/html/_sources/python/tree_utils.rst.txt @@ -0,0 +1,21 @@ +.. _utils: + +Tree Utils +========== + +In MLX we consider a python tree to be an arbitrarily nested collection of +dictionaries, lists and tuples without cycles. Functions in this module that +return python trees will be using the default python ``dict``, ``list`` and +``tuple`` but they can usually process objects that inherit from any of these. + +.. note:: + Dictionaries should have keys that are valid python identifiers. + +.. currentmodule:: mlx.utils + +.. autosummary:: + :toctree: _autosummary + + tree_flatten + tree_unflatten + tree_map diff --git a/docs/build/html/_sources/quick_start.rst.txt b/docs/build/html/_sources/quick_start.rst.txt new file mode 100644 index 000000000..c3e2b678b --- /dev/null +++ b/docs/build/html/_sources/quick_start.rst.txt @@ -0,0 +1,93 @@ +Quick Start Guide +================= + +MLX is a NumPy-like array framework designed for efficient and flexible +machine learning on Apple silicon. The Python API closely follows NumPy with +a few exceptions. MLX also has a fully featured C++ API which closely follows +the Python API. + +The main differences between MLX and NumPy are: + + - **Composable function transformations**: MLX has composable function + transformations for automatic differentiation, automatic vectorization, + and computation graph optimization. + - **Lazy computation**: Computations in MLX are lazy. Arrays are only + materialized when needed. + - **Multi-device**: Operations can run on any of the suppoorted devices (CPU, + GPU, ...) + +The design of MLX is strongly inspired by frameworks like `PyTorch +`_, `Jax `_, and +`ArrayFire `_. A noteable difference from these +frameworks and MLX is the *unified memory model*. Arrays in MLX live in shared +memory. Operations on MLX arrays can be performed on any of the supported +device types without performing data copies. Currently supported device types +are the CPU and GPU. + +Basics +------ + +.. currentmodule:: mlx.core + +Import ``mlx.core`` and make an :class:`array`: + +.. code-block:: python + + >> import mlx.core as mx + >> a = mx.array([1, 2, 3, 4]) + >> a.shape + [4] + >> a.dtype + int32 + >> b = mx.array([1.0, 2.0, 3.0, 4.0]) + >> b.dtype + float32 + +Operations in MLX are lazy. The outputs of MLX operations are not computed +until they are needed. To force an array to be evaluated use +:func:`eval`. Arrays will automatically be evaluated in a few cases. For +example, inspecting a scalar with :meth:`array.item`, printing an array, +or converting an array from :class:`array` to :class:`numpy.ndarray` all +automatically evaluate the array. + +.. code-block:: python + + >> c = a + b # c not yet evaluated + >> mx.eval(c) # evaluates c + >> c = a + b + >> print(c) # Also evaluates c + array([2, 4, 6, 8], dtype=float32) + >> c = a + b + >> import numpy as np + >> np.array(c) # Also evaluates c + array([2., 4., 6., 8.], dtype=float32) + +Function and Graph Transformations +---------------------------------- + +MLX has standard function transformations like :func:`grad` and :func:`vmap`. +Transformations can be composed arbitrarily. For example +``grad(vmap(grad(fn)))`` (or any other composition) is allowed. + +.. code-block:: python + + >> x = mx.array(0.0) + >> mx.sin(x) + array(0, dtype=float32) + >> mx.grad(mx.sin)(x) + array(1, dtype=float32) + >> mx.grad(mx.grad(mx.sin))(x) + array(-0, dtype=float32) + +Other gradient transformations include :func:`vjp` for vector-Jacobian products +and :func:`jvp` for Jacobian-vector products. + +Use :func:`value_and_grad` to efficiently compute both a function's output and +gradient with respect to the function's input. + + +Devices and Streams +------------------- + + + diff --git a/docs/build/html/_sources/using_streams.rst.txt b/docs/build/html/_sources/using_streams.rst.txt new file mode 100644 index 000000000..7b42e8d4b --- /dev/null +++ b/docs/build/html/_sources/using_streams.rst.txt @@ -0,0 +1,16 @@ +Using Streams +============= + +.. currentmodule:: mlx.core + +Specifying the :obj:`Stream` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +All operations (including random number generation) take an optional +keyword argument ``stream``. The ``stream`` kwarg specifies which +:obj:`Stream` the operation should run on. If the stream is unspecified then +the operation is run on the default stream of the default device: +``mx.default_stream(mx.default_device())``. The ``stream`` kwarg can also +be a :obj:`Device` (e.g. ``stream=my_device``) in which case the operation is +run on the default stream of the provided device +``mx.default_stream(my_device)``. diff --git a/docs/build/html/_static/_sphinx_javascript_frameworks_compat.js b/docs/build/html/_static/_sphinx_javascript_frameworks_compat.js new file mode 100644 index 000000000..8549469dc --- /dev/null +++ b/docs/build/html/_static/_sphinx_javascript_frameworks_compat.js @@ -0,0 +1,134 @@ +/* + * _sphinx_javascript_frameworks_compat.js + * ~~~~~~~~~~ + * + * Compatability shim for jQuery and underscores.js. + * + * WILL BE REMOVED IN Sphinx 6.0 + * xref RemovedInSphinx60Warning + * + */ + +/** + * select a different prefix for underscore + */ +$u = _.noConflict(); + + +/** + * small helper function to urldecode strings + * + * See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/decodeURIComponent#Decoding_query_parameters_from_a_URL + */ +jQuery.urldecode = function(x) { + if (!x) { + return x + } + return decodeURIComponent(x.replace(/\+/g, ' ')); +}; + +/** + * small helper function to urlencode strings + */ +jQuery.urlencode = encodeURIComponent; + +/** + * This function returns the parsed url parameters of the + * current request. Multiple values per key are supported, + * it will always return arrays of strings for the value parts. + */ +jQuery.getQueryParameters = function(s) { + if (typeof s === 'undefined') + s = document.location.search; + var parts = s.substr(s.indexOf('?') + 1).split('&'); + var result = {}; + for (var i = 0; i < parts.length; i++) { + var tmp = parts[i].split('=', 2); + var key = jQuery.urldecode(tmp[0]); + var value = jQuery.urldecode(tmp[1]); + if (key in result) + result[key].push(value); + else + result[key] = [value]; + } + return result; +}; + +/** + * highlight a given string on a jquery object by wrapping it in + * span elements with the given class name. + */ +jQuery.fn.highlightText = function(text, className) { + function highlight(node, addItems) { + if (node.nodeType === 3) { + var val = node.nodeValue; + var pos = val.toLowerCase().indexOf(text); + if (pos >= 0 && + !jQuery(node.parentNode).hasClass(className) && + !jQuery(node.parentNode).hasClass("nohighlight")) { + var span; + var isInSVG = jQuery(node).closest("body, svg, foreignObject").is("svg"); + if (isInSVG) { + span = document.createElementNS("http://www.w3.org/2000/svg", "tspan"); + } else { + span = document.createElement("span"); + span.className = className; + } + span.appendChild(document.createTextNode(val.substr(pos, text.length))); + node.parentNode.insertBefore(span, node.parentNode.insertBefore( + document.createTextNode(val.substr(pos + text.length)), + node.nextSibling)); + node.nodeValue = val.substr(0, pos); + if (isInSVG) { + var rect = document.createElementNS("http://www.w3.org/2000/svg", "rect"); + var bbox = node.parentElement.getBBox(); + rect.x.baseVal.value = bbox.x; + rect.y.baseVal.value = bbox.y; + rect.width.baseVal.value = bbox.width; + rect.height.baseVal.value = bbox.height; + rect.setAttribute('class', className); + addItems.push({ + "parent": node.parentNode, + "target": rect}); + } + } + } + else if (!jQuery(node).is("button, select, textarea")) { + jQuery.each(node.childNodes, function() { + highlight(this, addItems); + }); + } + } + var addItems = []; + var result = this.each(function() { + highlight(this, addItems); + }); + for (var i = 0; i < addItems.length; ++i) { + jQuery(addItems[i].parent).before(addItems[i].target); + } + return result; +}; + +/* + * backward compatibility for jQuery.browser + * This will be supported until firefox bug is fixed. + */ +if (!jQuery.browser) { + jQuery.uaMatch = function(ua) { + ua = ua.toLowerCase(); + + var match = /(chrome)[ \/]([\w.]+)/.exec(ua) || + /(webkit)[ \/]([\w.]+)/.exec(ua) || + /(opera)(?:.*version|)[ \/]([\w.]+)/.exec(ua) || + /(msie) ([\w.]+)/.exec(ua) || + ua.indexOf("compatible") < 0 && /(mozilla)(?:.*? rv:([\w.]+)|)/.exec(ua) || + []; + + return { + browser: match[ 1 ] || "", + version: match[ 2 ] || "0" + }; + }; + jQuery.browser = {}; + jQuery.browser[jQuery.uaMatch(navigator.userAgent).browser] = true; +} diff --git a/docs/build/html/_static/basic.css b/docs/build/html/_static/basic.css new file mode 100644 index 000000000..088967717 --- /dev/null +++ b/docs/build/html/_static/basic.css @@ -0,0 +1,930 @@ +/* + * basic.css + * ~~~~~~~~~ + * + * Sphinx stylesheet -- basic theme. + * + * :copyright: Copyright 2007-2022 by the Sphinx team, see AUTHORS. + * :license: BSD, see LICENSE for details. + * + */ + +/* -- main layout ----------------------------------------------------------- */ + +div.clearer { + clear: both; +} + +div.section::after { + display: block; + content: ''; + clear: left; +} + +/* -- relbar ---------------------------------------------------------------- */ + +div.related { + width: 100%; + font-size: 90%; +} + +div.related h3 { + display: none; +} + +div.related ul { + margin: 0; + padding: 0 0 0 10px; + list-style: none; +} + +div.related li { + display: inline; +} + +div.related li.right { + float: right; + margin-right: 5px; +} + +/* -- sidebar --------------------------------------------------------------- */ + +div.sphinxsidebarwrapper { + padding: 10px 5px 0 10px; +} + +div.sphinxsidebar { + float: left; + width: 230px; + margin-left: -100%; + font-size: 90%; + word-wrap: break-word; + overflow-wrap : break-word; +} + +div.sphinxsidebar ul { + list-style: none; +} + +div.sphinxsidebar ul ul, +div.sphinxsidebar ul.want-points { + margin-left: 20px; + list-style: square; +} + +div.sphinxsidebar ul ul { + margin-top: 0; + margin-bottom: 0; +} + +div.sphinxsidebar form { + margin-top: 10px; +} + +div.sphinxsidebar input { + border: 1px solid #98dbcc; + font-family: sans-serif; + font-size: 1em; +} + +div.sphinxsidebar #searchbox form.search { + overflow: hidden; +} + +div.sphinxsidebar #searchbox input[type="text"] { + float: left; + width: 80%; + padding: 0.25em; + box-sizing: border-box; +} + +div.sphinxsidebar #searchbox input[type="submit"] { + float: left; + width: 20%; + border-left: none; + padding: 0.25em; + box-sizing: border-box; +} + + +img { + border: 0; + max-width: 100%; +} + +/* -- search page ----------------------------------------------------------- */ + +ul.search { + margin: 10px 0 0 20px; + padding: 0; +} + +ul.search li { + padding: 5px 0 5px 20px; + background-image: url(file.png); + background-repeat: no-repeat; + background-position: 0 7px; +} + +ul.search li a { + font-weight: bold; +} + +ul.search li p.context { + color: #888; + margin: 2px 0 0 30px; + text-align: left; +} + +ul.keywordmatches li.goodmatch a { + font-weight: bold; +} + +/* -- index page ------------------------------------------------------------ */ + +table.contentstable { + width: 90%; + margin-left: auto; + margin-right: auto; +} + +table.contentstable p.biglink { + line-height: 150%; +} + +a.biglink { + font-size: 1.3em; +} + +span.linkdescr { + font-style: italic; + padding-top: 5px; + font-size: 90%; +} + +/* -- general index --------------------------------------------------------- */ + +table.indextable { + width: 100%; +} + +table.indextable td { + text-align: left; + vertical-align: top; +} + +table.indextable ul { + margin-top: 0; + margin-bottom: 0; + list-style-type: none; +} + +table.indextable > tbody > tr > td > ul { + padding-left: 0em; +} + +table.indextable tr.pcap { + height: 10px; +} + +table.indextable tr.cap { + margin-top: 10px; + background-color: #f2f2f2; +} + +img.toggler { + margin-right: 3px; + margin-top: 3px; + cursor: pointer; +} + +div.modindex-jumpbox { + border-top: 1px solid #ddd; + border-bottom: 1px solid #ddd; + margin: 1em 0 1em 0; + padding: 0.4em; +} + +div.genindex-jumpbox { + border-top: 1px solid #ddd; + border-bottom: 1px solid #ddd; + margin: 1em 0 1em 0; + padding: 0.4em; +} + +/* -- domain module index --------------------------------------------------- */ + +table.modindextable td { + padding: 2px; + border-collapse: collapse; +} + +/* -- general body styles --------------------------------------------------- */ + +div.body { + min-width: 360px; + max-width: 800px; +} + +div.body p, div.body dd, div.body li, div.body blockquote { + -moz-hyphens: auto; + -ms-hyphens: auto; + -webkit-hyphens: auto; + hyphens: auto; +} + +a.headerlink { + visibility: hidden; +} + +h1:hover > a.headerlink, +h2:hover > a.headerlink, +h3:hover > a.headerlink, +h4:hover > a.headerlink, +h5:hover > a.headerlink, +h6:hover > a.headerlink, +dt:hover > a.headerlink, +caption:hover > a.headerlink, +p.caption:hover > a.headerlink, +div.code-block-caption:hover > a.headerlink { + visibility: visible; +} + +div.body p.caption { + text-align: inherit; +} + +div.body td { + text-align: left; +} + +.first { + margin-top: 0 !important; +} + +p.rubric { + margin-top: 30px; + font-weight: bold; +} + +img.align-left, figure.align-left, .figure.align-left, object.align-left { + clear: left; + float: left; + margin-right: 1em; +} + +img.align-right, figure.align-right, .figure.align-right, object.align-right { + clear: right; + float: right; + margin-left: 1em; +} + +img.align-center, figure.align-center, .figure.align-center, object.align-center { + display: block; + margin-left: auto; + margin-right: auto; +} + +img.align-default, figure.align-default, .figure.align-default { + display: block; + margin-left: auto; + margin-right: auto; +} + +.align-left { + text-align: left; +} + +.align-center { + text-align: center; +} + +.align-default { + text-align: center; +} + +.align-right { + text-align: right; +} + +/* -- sidebars -------------------------------------------------------------- */ + +div.sidebar, +aside.sidebar { + margin: 0 0 0.5em 1em; + border: 1px solid #ddb; + padding: 7px; + background-color: #ffe; + width: 40%; + float: right; + clear: right; + overflow-x: auto; +} + +p.sidebar-title { + font-weight: bold; +} +nav.contents, +aside.topic, + +div.admonition, div.topic, blockquote { + clear: left; +} + +/* -- topics ---------------------------------------------------------------- */ +nav.contents, +aside.topic, + +div.topic { + border: 1px solid #ccc; + padding: 7px; + margin: 10px 0 10px 0; +} + +p.topic-title { + font-size: 1.1em; + font-weight: bold; + margin-top: 10px; +} + +/* -- admonitions ----------------------------------------------------------- */ + +div.admonition { + margin-top: 10px; + margin-bottom: 10px; + padding: 7px; +} + +div.admonition dt { + font-weight: bold; +} + +p.admonition-title { + margin: 0px 10px 5px 0px; + font-weight: bold; +} + +div.body p.centered { + text-align: center; + margin-top: 25px; +} + +/* -- content of sidebars/topics/admonitions -------------------------------- */ + +div.sidebar > :last-child, +aside.sidebar > :last-child, +nav.contents > :last-child, +aside.topic > :last-child, + +div.topic > :last-child, +div.admonition > :last-child { + margin-bottom: 0; +} + +div.sidebar::after, +aside.sidebar::after, +nav.contents::after, +aside.topic::after, + +div.topic::after, +div.admonition::after, +blockquote::after { + display: block; + content: ''; + clear: both; +} + +/* -- tables ---------------------------------------------------------------- */ + +table.docutils { + margin-top: 10px; + margin-bottom: 10px; + border: 0; + border-collapse: collapse; +} + +table.align-center { + margin-left: auto; + margin-right: auto; +} + +table.align-default { + margin-left: auto; + margin-right: auto; +} + +table caption span.caption-number { + font-style: italic; +} + +table caption span.caption-text { +} + +table.docutils td, table.docutils th { + padding: 1px 8px 1px 5px; + border-top: 0; + border-left: 0; + border-right: 0; + border-bottom: 1px solid #aaa; +} + +th { + text-align: left; + padding-right: 5px; +} + +table.citation { + border-left: solid 1px gray; + margin-left: 1px; +} + +table.citation td { + border-bottom: none; +} + +th > :first-child, +td > :first-child { + margin-top: 0px; +} + +th > :last-child, +td > :last-child { + margin-bottom: 0px; +} + +/* -- figures --------------------------------------------------------------- */ + +div.figure, figure { + margin: 0.5em; + padding: 0.5em; +} + +div.figure p.caption, figcaption { + padding: 0.3em; +} + +div.figure p.caption span.caption-number, +figcaption span.caption-number { + font-style: italic; +} + +div.figure p.caption span.caption-text, +figcaption span.caption-text { +} + +/* -- field list styles ----------------------------------------------------- */ + +table.field-list td, table.field-list th { + border: 0 !important; +} + +.field-list ul { + margin: 0; + padding-left: 1em; +} + +.field-list p { + margin: 0; +} + +.field-name { + -moz-hyphens: manual; + -ms-hyphens: manual; + -webkit-hyphens: manual; + hyphens: manual; +} + +/* -- hlist styles ---------------------------------------------------------- */ + +table.hlist { + margin: 1em 0; +} + +table.hlist td { + vertical-align: top; +} + +/* -- object description styles --------------------------------------------- */ + +.sig { + font-family: 'Consolas', 'Menlo', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', monospace; +} + +.sig-name, code.descname { + background-color: transparent; + font-weight: bold; +} + +.sig-name { + font-size: 1.1em; +} + +code.descname { + font-size: 1.2em; +} + +.sig-prename, code.descclassname { + background-color: transparent; +} + +.optional { + font-size: 1.3em; +} + +.sig-paren { + font-size: larger; +} + +.sig-param.n { + font-style: italic; +} + +/* C++ specific styling */ + +.sig-inline.c-texpr, +.sig-inline.cpp-texpr { + font-family: unset; +} + +.sig.c .k, .sig.c .kt, +.sig.cpp .k, .sig.cpp .kt { + color: #0033B3; +} + +.sig.c .m, +.sig.cpp .m { + color: #1750EB; +} + +.sig.c .s, .sig.c .sc, +.sig.cpp .s, .sig.cpp .sc { + color: #067D17; +} + + +/* -- other body styles ----------------------------------------------------- */ + +ol.arabic { + list-style: decimal; +} + +ol.loweralpha { + list-style: lower-alpha; +} + +ol.upperalpha { + list-style: upper-alpha; +} + +ol.lowerroman { + list-style: lower-roman; +} + +ol.upperroman { + list-style: upper-roman; +} + +:not(li) > ol > li:first-child > :first-child, +:not(li) > ul > li:first-child > :first-child { + margin-top: 0px; +} + +:not(li) > ol > li:last-child > :last-child, +:not(li) > ul > li:last-child > :last-child { + margin-bottom: 0px; +} + +ol.simple ol p, +ol.simple ul p, +ul.simple ol p, +ul.simple ul p { + margin-top: 0; +} + +ol.simple > li:not(:first-child) > p, +ul.simple > li:not(:first-child) > p { + margin-top: 0; +} + +ol.simple p, +ul.simple p { + margin-bottom: 0; +} + +/* Docutils 0.17 and older (footnotes & citations) */ +dl.footnote > dt, +dl.citation > dt { + float: left; + margin-right: 0.5em; +} + +dl.footnote > dd, +dl.citation > dd { + margin-bottom: 0em; +} + +dl.footnote > dd:after, +dl.citation > dd:after { + content: ""; + clear: both; +} + +/* Docutils 0.18+ (footnotes & citations) */ +aside.footnote > span, +div.citation > span { + float: left; +} +aside.footnote > span:last-of-type, +div.citation > span:last-of-type { + padding-right: 0.5em; +} +aside.footnote > p { + margin-left: 2em; +} +div.citation > p { + margin-left: 4em; +} +aside.footnote > p:last-of-type, +div.citation > p:last-of-type { + margin-bottom: 0em; +} +aside.footnote > p:last-of-type:after, +div.citation > p:last-of-type:after { + content: ""; + clear: both; +} + +/* Footnotes & citations ends */ + +dl.field-list { + display: grid; + grid-template-columns: fit-content(30%) auto; +} + +dl.field-list > dt { + font-weight: bold; + word-break: break-word; + padding-left: 0.5em; + padding-right: 5px; +} + +dl.field-list > dt:after { + content: ":"; +} + +dl.field-list > dd { + padding-left: 0.5em; + margin-top: 0em; + margin-left: 0em; + margin-bottom: 0em; +} + +dl { + margin-bottom: 15px; +} + +dd > :first-child { + margin-top: 0px; +} + +dd ul, dd table { + margin-bottom: 10px; +} + +dd { + margin-top: 3px; + margin-bottom: 10px; + margin-left: 30px; +} + +dl > dd:last-child, +dl > dd:last-child > :last-child { + margin-bottom: 0; +} + +dt:target, span.highlighted { + background-color: #fbe54e; +} + +rect.highlighted { + fill: #fbe54e; +} + +dl.glossary dt { + font-weight: bold; + font-size: 1.1em; +} + +.versionmodified { + font-style: italic; +} + +.system-message { + background-color: #fda; + padding: 5px; + border: 3px solid red; +} + +.footnote:target { + background-color: #ffa; +} + +.line-block { + display: block; + margin-top: 1em; + margin-bottom: 1em; +} + +.line-block .line-block { + margin-top: 0; + margin-bottom: 0; + margin-left: 1.5em; +} + +.guilabel, .menuselection { + font-family: sans-serif; +} + +.accelerator { + text-decoration: underline; +} + +.classifier { + font-style: oblique; +} + +.classifier:before { + font-style: normal; + margin: 0 0.5em; + content: ":"; + display: inline-block; +} + +abbr, acronym { + border-bottom: dotted 1px; + cursor: help; +} + +/* -- code displays --------------------------------------------------------- */ + +pre { + overflow: auto; + overflow-y: hidden; /* fixes display issues on Chrome browsers */ +} + +pre, div[class*="highlight-"] { + clear: both; +} + +span.pre { + -moz-hyphens: none; + -ms-hyphens: none; + -webkit-hyphens: none; + hyphens: none; + white-space: nowrap; +} + +div[class*="highlight-"] { + margin: 1em 0; +} + +td.linenos pre { + border: 0; + background-color: transparent; + color: #aaa; +} + +table.highlighttable { + display: block; +} + +table.highlighttable tbody { + display: block; +} + +table.highlighttable tr { + display: flex; +} + +table.highlighttable td { + margin: 0; + padding: 0; +} + +table.highlighttable td.linenos { + padding-right: 0.5em; +} + +table.highlighttable td.code { + flex: 1; + overflow: hidden; +} + +.highlight .hll { + display: block; +} + +div.highlight pre, +table.highlighttable pre { + margin: 0; +} + +div.code-block-caption + div { + margin-top: 0; +} + +div.code-block-caption { + margin-top: 1em; + padding: 2px 5px; + font-size: small; +} + +div.code-block-caption code { + background-color: transparent; +} + +table.highlighttable td.linenos, +span.linenos, +div.highlight span.gp { /* gp: Generic.Prompt */ + user-select: none; + -webkit-user-select: text; /* Safari fallback only */ + -webkit-user-select: none; /* Chrome/Safari */ + -moz-user-select: none; /* Firefox */ + -ms-user-select: none; /* IE10+ */ +} + +div.code-block-caption span.caption-number { + padding: 0.1em 0.3em; + font-style: italic; +} + +div.code-block-caption span.caption-text { +} + +div.literal-block-wrapper { + margin: 1em 0; +} + +code.xref, a code { + background-color: transparent; + font-weight: bold; +} + +h1 code, h2 code, h3 code, h4 code, h5 code, h6 code { + background-color: transparent; +} + +.viewcode-link { + float: right; +} + +.viewcode-back { + float: right; + font-family: sans-serif; +} + +div.viewcode-block:target { + margin: -1px -10px; + padding: 0 10px; +} + +/* -- math display ---------------------------------------------------------- */ + +img.math { + vertical-align: middle; +} + +div.body div.math p { + text-align: center; +} + +span.eqno { + float: right; +} + +span.eqno a.headerlink { + position: absolute; + z-index: 1; +} + +div.math:hover a.headerlink { + visibility: visible; +} + +/* -- printout stylesheet --------------------------------------------------- */ + +@media print { + div.document, + div.documentwrapper, + div.bodywrapper { + margin: 0 !important; + width: 100%; + } + + div.sphinxsidebar, + div.related, + div.footer, + #top-link { + display: none; + } +} \ No newline at end of file diff --git a/docs/build/html/_static/css/badge_only.css b/docs/build/html/_static/css/badge_only.css new file mode 100644 index 000000000..c718cee44 --- /dev/null +++ b/docs/build/html/_static/css/badge_only.css @@ -0,0 +1 @@ +.clearfix{*zoom:1}.clearfix:after,.clearfix:before{display:table;content:""}.clearfix:after{clear:both}@font-face{font-family:FontAwesome;font-style:normal;font-weight:400;src:url(fonts/fontawesome-webfont.eot?674f50d287a8c48dc19ba404d20fe713?#iefix) format("embedded-opentype"),url(fonts/fontawesome-webfont.woff2?af7ae505a9eed503f8b8e6982036873e) format("woff2"),url(fonts/fontawesome-webfont.woff?fee66e712a8a08eef5805a46892932ad) format("woff"),url(fonts/fontawesome-webfont.ttf?b06871f281fee6b241d60582ae9369b9) format("truetype"),url(fonts/fontawesome-webfont.svg?912ec66d7572ff821749319396470bde#FontAwesome) format("svg")}.fa:before{font-family:FontAwesome;font-style:normal;font-weight:400;line-height:1}.fa:before,a .fa{text-decoration:inherit}.fa:before,a .fa,li .fa{display:inline-block}li .fa-large:before{width:1.875em}ul.fas{list-style-type:none;margin-left:2em;text-indent:-.8em}ul.fas li .fa{width:.8em}ul.fas li .fa-large:before{vertical-align:baseline}.fa-book:before,.icon-book:before{content:"\f02d"}.fa-caret-down:before,.icon-caret-down:before{content:"\f0d7"}.fa-caret-up:before,.icon-caret-up:before{content:"\f0d8"}.fa-caret-left:before,.icon-caret-left:before{content:"\f0d9"}.fa-caret-right:before,.icon-caret-right:before{content:"\f0da"}.rst-versions{position:fixed;bottom:0;left:0;width:300px;color:#fcfcfc;background:#1f1d1d;font-family:Lato,proxima-nova,Helvetica Neue,Arial,sans-serif;z-index:400}.rst-versions a{color:#2980b9;text-decoration:none}.rst-versions .rst-badge-small{display:none}.rst-versions .rst-current-version{padding:12px;background-color:#272525;display:block;text-align:right;font-size:90%;cursor:pointer;color:#27ae60}.rst-versions .rst-current-version:after{clear:both;content:"";display:block}.rst-versions .rst-current-version .fa{color:#fcfcfc}.rst-versions .rst-current-version .fa-book,.rst-versions .rst-current-version .icon-book{float:left}.rst-versions .rst-current-version.rst-out-of-date{background-color:#e74c3c;color:#fff}.rst-versions .rst-current-version.rst-active-old-version{background-color:#f1c40f;color:#000}.rst-versions.shift-up{height:auto;max-height:100%;overflow-y:scroll}.rst-versions.shift-up .rst-other-versions{display:block}.rst-versions .rst-other-versions{font-size:90%;padding:12px;color:grey;display:none}.rst-versions .rst-other-versions hr{display:block;height:1px;border:0;margin:20px 0;padding:0;border-top:1px solid #413d3d}.rst-versions .rst-other-versions dd{display:inline-block;margin:0}.rst-versions .rst-other-versions dd a{display:inline-block;padding:6px;color:#fcfcfc}.rst-versions.rst-badge{width:auto;bottom:20px;right:20px;left:auto;border:none;max-width:300px;max-height:90%}.rst-versions.rst-badge .fa-book,.rst-versions.rst-badge .icon-book{float:none;line-height:30px}.rst-versions.rst-badge.shift-up .rst-current-version{text-align:right}.rst-versions.rst-badge.shift-up .rst-current-version .fa-book,.rst-versions.rst-badge.shift-up .rst-current-version .icon-book{float:left}.rst-versions.rst-badge>.rst-current-version{width:auto;height:30px;line-height:30px;padding:0 6px;display:block;text-align:center}@media screen and (max-width:768px){.rst-versions{width:85%;display:none}.rst-versions.shift{display:block}} \ No newline at end of file diff --git a/docs/build/html/_static/css/fonts/Roboto-Slab-Bold.woff b/docs/build/html/_static/css/fonts/Roboto-Slab-Bold.woff new file mode 100644 index 000000000..6cb600001 Binary files /dev/null and b/docs/build/html/_static/css/fonts/Roboto-Slab-Bold.woff differ diff --git a/docs/build/html/_static/css/fonts/Roboto-Slab-Bold.woff2 b/docs/build/html/_static/css/fonts/Roboto-Slab-Bold.woff2 new file mode 100644 index 000000000..7059e2314 Binary files /dev/null and b/docs/build/html/_static/css/fonts/Roboto-Slab-Bold.woff2 differ diff --git a/docs/build/html/_static/css/fonts/Roboto-Slab-Regular.woff b/docs/build/html/_static/css/fonts/Roboto-Slab-Regular.woff new file mode 100644 index 000000000..f815f63f9 Binary files /dev/null and b/docs/build/html/_static/css/fonts/Roboto-Slab-Regular.woff differ diff --git a/docs/build/html/_static/css/fonts/Roboto-Slab-Regular.woff2 b/docs/build/html/_static/css/fonts/Roboto-Slab-Regular.woff2 new file mode 100644 index 000000000..f2c76e5bd Binary files /dev/null and b/docs/build/html/_static/css/fonts/Roboto-Slab-Regular.woff2 differ diff --git a/docs/build/html/_static/css/fonts/fontawesome-webfont.eot b/docs/build/html/_static/css/fonts/fontawesome-webfont.eot new file mode 100644 index 000000000..e9f60ca95 Binary files /dev/null and b/docs/build/html/_static/css/fonts/fontawesome-webfont.eot differ diff --git a/docs/build/html/_static/css/fonts/fontawesome-webfont.svg b/docs/build/html/_static/css/fonts/fontawesome-webfont.svg new file mode 100644 index 000000000..855c845e5 --- /dev/null +++ b/docs/build/html/_static/css/fonts/fontawesome-webfont.svg @@ -0,0 +1,2671 @@ + + + + +Created by FontForge 20120731 at Mon Oct 24 17:37:40 2016 + By ,,, +Copyright Dave Gandy 2016. All rights reserved. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/build/html/_static/css/fonts/fontawesome-webfont.ttf b/docs/build/html/_static/css/fonts/fontawesome-webfont.ttf new file mode 100644 index 000000000..35acda2fa Binary files /dev/null and b/docs/build/html/_static/css/fonts/fontawesome-webfont.ttf differ diff --git a/docs/build/html/_static/css/fonts/fontawesome-webfont.woff b/docs/build/html/_static/css/fonts/fontawesome-webfont.woff new file mode 100644 index 000000000..400014a4b Binary files /dev/null and b/docs/build/html/_static/css/fonts/fontawesome-webfont.woff differ diff --git a/docs/build/html/_static/css/fonts/fontawesome-webfont.woff2 b/docs/build/html/_static/css/fonts/fontawesome-webfont.woff2 new file mode 100644 index 000000000..4d13fc604 Binary files /dev/null and b/docs/build/html/_static/css/fonts/fontawesome-webfont.woff2 differ diff --git a/docs/build/html/_static/css/fonts/lato-bold-italic.woff b/docs/build/html/_static/css/fonts/lato-bold-italic.woff new file mode 100644 index 000000000..88ad05b9f Binary files /dev/null and b/docs/build/html/_static/css/fonts/lato-bold-italic.woff differ diff --git a/docs/build/html/_static/css/fonts/lato-bold-italic.woff2 b/docs/build/html/_static/css/fonts/lato-bold-italic.woff2 new file mode 100644 index 000000000..c4e3d804b Binary files /dev/null and b/docs/build/html/_static/css/fonts/lato-bold-italic.woff2 differ diff --git a/docs/build/html/_static/css/fonts/lato-bold.woff b/docs/build/html/_static/css/fonts/lato-bold.woff new file mode 100644 index 000000000..c6dff51f0 Binary files /dev/null and b/docs/build/html/_static/css/fonts/lato-bold.woff differ diff --git a/docs/build/html/_static/css/fonts/lato-bold.woff2 b/docs/build/html/_static/css/fonts/lato-bold.woff2 new file mode 100644 index 000000000..bb195043c Binary files /dev/null and b/docs/build/html/_static/css/fonts/lato-bold.woff2 differ diff --git a/docs/build/html/_static/css/fonts/lato-normal-italic.woff b/docs/build/html/_static/css/fonts/lato-normal-italic.woff new file mode 100644 index 000000000..76114bc03 Binary files /dev/null and b/docs/build/html/_static/css/fonts/lato-normal-italic.woff differ diff --git a/docs/build/html/_static/css/fonts/lato-normal-italic.woff2 b/docs/build/html/_static/css/fonts/lato-normal-italic.woff2 new file mode 100644 index 000000000..3404f37e2 Binary files /dev/null and b/docs/build/html/_static/css/fonts/lato-normal-italic.woff2 differ diff --git a/docs/build/html/_static/css/fonts/lato-normal.woff b/docs/build/html/_static/css/fonts/lato-normal.woff new file mode 100644 index 000000000..ae1307ff5 Binary files /dev/null and b/docs/build/html/_static/css/fonts/lato-normal.woff differ diff --git a/docs/build/html/_static/css/fonts/lato-normal.woff2 b/docs/build/html/_static/css/fonts/lato-normal.woff2 new file mode 100644 index 000000000..3bf984332 Binary files /dev/null and b/docs/build/html/_static/css/fonts/lato-normal.woff2 differ diff --git a/docs/build/html/_static/css/theme.css b/docs/build/html/_static/css/theme.css new file mode 100644 index 000000000..19a446a0e --- /dev/null +++ b/docs/build/html/_static/css/theme.css @@ -0,0 +1,4 @@ +html{box-sizing:border-box}*,:after,:before{box-sizing:inherit}article,aside,details,figcaption,figure,footer,header,hgroup,nav,section{display:block}audio,canvas,video{display:inline-block;*display:inline;*zoom:1}[hidden],audio:not([controls]){display:none}*{-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}html{font-size:100%;-webkit-text-size-adjust:100%;-ms-text-size-adjust:100%}body{margin:0}a:active,a:hover{outline:0}abbr[title]{border-bottom:1px dotted}b,strong{font-weight:700}blockquote{margin:0}dfn{font-style:italic}ins{background:#ff9;text-decoration:none}ins,mark{color:#000}mark{background:#ff0;font-style:italic;font-weight:700}.rst-content code,.rst-content tt,code,kbd,pre,samp{font-family:monospace,serif;_font-family:courier new,monospace;font-size:1em}pre{white-space:pre}q{quotes:none}q:after,q:before{content:"";content:none}small{font-size:85%}sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}sup{top:-.5em}sub{bottom:-.25em}dl,ol,ul{margin:0;padding:0;list-style:none;list-style-image:none}li{list-style:none}dd{margin:0}img{border:0;-ms-interpolation-mode:bicubic;vertical-align:middle;max-width:100%}svg:not(:root){overflow:hidden}figure,form{margin:0}label{cursor:pointer}button,input,select,textarea{font-size:100%;margin:0;vertical-align:baseline;*vertical-align:middle}button,input{line-height:normal}button,input[type=button],input[type=reset],input[type=submit]{cursor:pointer;-webkit-appearance:button;*overflow:visible}button[disabled],input[disabled]{cursor:default}input[type=search]{-webkit-appearance:textfield;-moz-box-sizing:content-box;-webkit-box-sizing:content-box;box-sizing:content-box}textarea{resize:vertical}table{border-collapse:collapse;border-spacing:0}td{vertical-align:top}.chromeframe{margin:.2em 0;background:#ccc;color:#000;padding:.2em 0}.ir{display:block;border:0;text-indent:-999em;overflow:hidden;background-color:transparent;background-repeat:no-repeat;text-align:left;direction:ltr;*line-height:0}.ir br{display:none}.hidden{display:none!important;visibility:hidden}.visuallyhidden{border:0;clip:rect(0 0 0 0);height:1px;margin:-1px;overflow:hidden;padding:0;position:absolute;width:1px}.visuallyhidden.focusable:active,.visuallyhidden.focusable:focus{clip:auto;height:auto;margin:0;overflow:visible;position:static;width:auto}.invisible{visibility:hidden}.relative{position:relative}big,small{font-size:100%}@media print{body,html,section{background:none!important}*{box-shadow:none!important;text-shadow:none!important;filter:none!important;-ms-filter:none!important}a,a:visited{text-decoration:underline}.ir a:after,a[href^="#"]:after,a[href^="javascript:"]:after{content:""}blockquote,pre{page-break-inside:avoid}thead{display:table-header-group}img,tr{page-break-inside:avoid}img{max-width:100%!important}@page{margin:.5cm}.rst-content .toctree-wrapper>p.caption,h2,h3,p{orphans:3;widows:3}.rst-content .toctree-wrapper>p.caption,h2,h3{page-break-after:avoid}}.btn,.fa:before,.icon:before,.rst-content .admonition,.rst-content .admonition-title:before,.rst-content .admonition-todo,.rst-content .attention,.rst-content .caution,.rst-content .code-block-caption .headerlink:before,.rst-content .danger,.rst-content .eqno .headerlink:before,.rst-content .error,.rst-content .hint,.rst-content .important,.rst-content .note,.rst-content .seealso,.rst-content .tip,.rst-content .warning,.rst-content code.download span:first-child:before,.rst-content dl dt .headerlink:before,.rst-content h1 .headerlink:before,.rst-content h2 .headerlink:before,.rst-content h3 .headerlink:before,.rst-content h4 .headerlink:before,.rst-content h5 .headerlink:before,.rst-content h6 .headerlink:before,.rst-content p.caption .headerlink:before,.rst-content p .headerlink:before,.rst-content table>caption .headerlink:before,.rst-content tt.download span:first-child:before,.wy-alert,.wy-dropdown .caret:before,.wy-inline-validate.wy-inline-validate-danger .wy-input-context:before,.wy-inline-validate.wy-inline-validate-info .wy-input-context:before,.wy-inline-validate.wy-inline-validate-success .wy-input-context:before,.wy-inline-validate.wy-inline-validate-warning .wy-input-context:before,.wy-menu-vertical li.current>a button.toctree-expand:before,.wy-menu-vertical li.on a button.toctree-expand:before,.wy-menu-vertical li button.toctree-expand:before,input[type=color],input[type=date],input[type=datetime-local],input[type=datetime],input[type=email],input[type=month],input[type=number],input[type=password],input[type=search],input[type=tel],input[type=text],input[type=time],input[type=url],input[type=week],select,textarea{-webkit-font-smoothing:antialiased}.clearfix{*zoom:1}.clearfix:after,.clearfix:before{display:table;content:""}.clearfix:after{clear:both}/*! + * Font Awesome 4.7.0 by @davegandy - http://fontawesome.io - @fontawesome + * License - http://fontawesome.io/license (Font: SIL OFL 1.1, CSS: MIT License) + */@font-face{font-family:FontAwesome;src:url(fonts/fontawesome-webfont.eot?674f50d287a8c48dc19ba404d20fe713);src:url(fonts/fontawesome-webfont.eot?674f50d287a8c48dc19ba404d20fe713?#iefix&v=4.7.0) format("embedded-opentype"),url(fonts/fontawesome-webfont.woff2?af7ae505a9eed503f8b8e6982036873e) format("woff2"),url(fonts/fontawesome-webfont.woff?fee66e712a8a08eef5805a46892932ad) format("woff"),url(fonts/fontawesome-webfont.ttf?b06871f281fee6b241d60582ae9369b9) format("truetype"),url(fonts/fontawesome-webfont.svg?912ec66d7572ff821749319396470bde#fontawesomeregular) format("svg");font-weight:400;font-style:normal}.fa,.icon,.rst-content .admonition-title,.rst-content .code-block-caption .headerlink,.rst-content .eqno .headerlink,.rst-content code.download span:first-child,.rst-content dl dt .headerlink,.rst-content h1 .headerlink,.rst-content h2 .headerlink,.rst-content h3 .headerlink,.rst-content h4 .headerlink,.rst-content h5 .headerlink,.rst-content h6 .headerlink,.rst-content p.caption .headerlink,.rst-content p .headerlink,.rst-content table>caption .headerlink,.rst-content tt.download span:first-child,.wy-menu-vertical li.current>a button.toctree-expand,.wy-menu-vertical li.on a button.toctree-expand,.wy-menu-vertical li button.toctree-expand{display:inline-block;font:normal normal normal 14px/1 FontAwesome;font-size:inherit;text-rendering:auto;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale}.fa-lg{font-size:1.33333em;line-height:.75em;vertical-align:-15%}.fa-2x{font-size:2em}.fa-3x{font-size:3em}.fa-4x{font-size:4em}.fa-5x{font-size:5em}.fa-fw{width:1.28571em;text-align:center}.fa-ul{padding-left:0;margin-left:2.14286em;list-style-type:none}.fa-ul>li{position:relative}.fa-li{position:absolute;left:-2.14286em;width:2.14286em;top:.14286em;text-align:center}.fa-li.fa-lg{left:-1.85714em}.fa-border{padding:.2em .25em .15em;border:.08em solid #eee;border-radius:.1em}.fa-pull-left{float:left}.fa-pull-right{float:right}.fa-pull-left.icon,.fa.fa-pull-left,.rst-content .code-block-caption .fa-pull-left.headerlink,.rst-content .eqno .fa-pull-left.headerlink,.rst-content .fa-pull-left.admonition-title,.rst-content code.download span.fa-pull-left:first-child,.rst-content dl dt .fa-pull-left.headerlink,.rst-content h1 .fa-pull-left.headerlink,.rst-content h2 .fa-pull-left.headerlink,.rst-content h3 .fa-pull-left.headerlink,.rst-content h4 .fa-pull-left.headerlink,.rst-content h5 .fa-pull-left.headerlink,.rst-content h6 .fa-pull-left.headerlink,.rst-content p .fa-pull-left.headerlink,.rst-content table>caption .fa-pull-left.headerlink,.rst-content tt.download span.fa-pull-left:first-child,.wy-menu-vertical li.current>a button.fa-pull-left.toctree-expand,.wy-menu-vertical li.on a button.fa-pull-left.toctree-expand,.wy-menu-vertical li button.fa-pull-left.toctree-expand{margin-right:.3em}.fa-pull-right.icon,.fa.fa-pull-right,.rst-content .code-block-caption .fa-pull-right.headerlink,.rst-content .eqno .fa-pull-right.headerlink,.rst-content .fa-pull-right.admonition-title,.rst-content code.download span.fa-pull-right:first-child,.rst-content dl dt .fa-pull-right.headerlink,.rst-content h1 .fa-pull-right.headerlink,.rst-content h2 .fa-pull-right.headerlink,.rst-content h3 .fa-pull-right.headerlink,.rst-content h4 .fa-pull-right.headerlink,.rst-content h5 .fa-pull-right.headerlink,.rst-content h6 .fa-pull-right.headerlink,.rst-content p .fa-pull-right.headerlink,.rst-content table>caption .fa-pull-right.headerlink,.rst-content tt.download span.fa-pull-right:first-child,.wy-menu-vertical li.current>a button.fa-pull-right.toctree-expand,.wy-menu-vertical li.on a button.fa-pull-right.toctree-expand,.wy-menu-vertical li button.fa-pull-right.toctree-expand{margin-left:.3em}.pull-right{float:right}.pull-left{float:left}.fa.pull-left,.pull-left.icon,.rst-content .code-block-caption .pull-left.headerlink,.rst-content .eqno .pull-left.headerlink,.rst-content .pull-left.admonition-title,.rst-content code.download span.pull-left:first-child,.rst-content dl dt .pull-left.headerlink,.rst-content h1 .pull-left.headerlink,.rst-content h2 .pull-left.headerlink,.rst-content h3 .pull-left.headerlink,.rst-content h4 .pull-left.headerlink,.rst-content h5 .pull-left.headerlink,.rst-content h6 .pull-left.headerlink,.rst-content p .pull-left.headerlink,.rst-content table>caption .pull-left.headerlink,.rst-content tt.download span.pull-left:first-child,.wy-menu-vertical li.current>a button.pull-left.toctree-expand,.wy-menu-vertical li.on a button.pull-left.toctree-expand,.wy-menu-vertical li button.pull-left.toctree-expand{margin-right:.3em}.fa.pull-right,.pull-right.icon,.rst-content .code-block-caption .pull-right.headerlink,.rst-content .eqno .pull-right.headerlink,.rst-content .pull-right.admonition-title,.rst-content code.download span.pull-right:first-child,.rst-content dl dt .pull-right.headerlink,.rst-content h1 .pull-right.headerlink,.rst-content h2 .pull-right.headerlink,.rst-content h3 .pull-right.headerlink,.rst-content h4 .pull-right.headerlink,.rst-content h5 .pull-right.headerlink,.rst-content h6 .pull-right.headerlink,.rst-content p .pull-right.headerlink,.rst-content table>caption .pull-right.headerlink,.rst-content tt.download span.pull-right:first-child,.wy-menu-vertical li.current>a button.pull-right.toctree-expand,.wy-menu-vertical li.on a button.pull-right.toctree-expand,.wy-menu-vertical li button.pull-right.toctree-expand{margin-left:.3em}.fa-spin{-webkit-animation:fa-spin 2s linear infinite;animation:fa-spin 2s linear infinite}.fa-pulse{-webkit-animation:fa-spin 1s steps(8) infinite;animation:fa-spin 1s steps(8) infinite}@-webkit-keyframes fa-spin{0%{-webkit-transform:rotate(0deg);transform:rotate(0deg)}to{-webkit-transform:rotate(359deg);transform:rotate(359deg)}}@keyframes fa-spin{0%{-webkit-transform:rotate(0deg);transform:rotate(0deg)}to{-webkit-transform:rotate(359deg);transform:rotate(359deg)}}.fa-rotate-90{-ms-filter:"progid:DXImageTransform.Microsoft.BasicImage(rotation=1)";-webkit-transform:rotate(90deg);-ms-transform:rotate(90deg);transform:rotate(90deg)}.fa-rotate-180{-ms-filter:"progid:DXImageTransform.Microsoft.BasicImage(rotation=2)";-webkit-transform:rotate(180deg);-ms-transform:rotate(180deg);transform:rotate(180deg)}.fa-rotate-270{-ms-filter:"progid:DXImageTransform.Microsoft.BasicImage(rotation=3)";-webkit-transform:rotate(270deg);-ms-transform:rotate(270deg);transform:rotate(270deg)}.fa-flip-horizontal{-ms-filter:"progid:DXImageTransform.Microsoft.BasicImage(rotation=0, mirror=1)";-webkit-transform:scaleX(-1);-ms-transform:scaleX(-1);transform:scaleX(-1)}.fa-flip-vertical{-ms-filter:"progid:DXImageTransform.Microsoft.BasicImage(rotation=2, mirror=1)";-webkit-transform:scaleY(-1);-ms-transform:scaleY(-1);transform:scaleY(-1)}:root .fa-flip-horizontal,:root .fa-flip-vertical,:root .fa-rotate-90,:root .fa-rotate-180,:root .fa-rotate-270{filter:none}.fa-stack{position:relative;display:inline-block;width:2em;height:2em;line-height:2em;vertical-align:middle}.fa-stack-1x,.fa-stack-2x{position:absolute;left:0;width:100%;text-align:center}.fa-stack-1x{line-height:inherit}.fa-stack-2x{font-size:2em}.fa-inverse{color:#fff}.fa-glass:before{content:""}.fa-music:before{content:""}.fa-search:before,.icon-search:before{content:""}.fa-envelope-o:before{content:""}.fa-heart:before{content:""}.fa-star:before{content:""}.fa-star-o:before{content:""}.fa-user:before{content:""}.fa-film:before{content:""}.fa-th-large:before{content:""}.fa-th:before{content:""}.fa-th-list:before{content:""}.fa-check:before{content:""}.fa-close:before,.fa-remove:before,.fa-times:before{content:""}.fa-search-plus:before{content:""}.fa-search-minus:before{content:""}.fa-power-off:before{content:""}.fa-signal:before{content:""}.fa-cog:before,.fa-gear:before{content:""}.fa-trash-o:before{content:""}.fa-home:before,.icon-home:before{content:""}.fa-file-o:before{content:""}.fa-clock-o:before{content:""}.fa-road:before{content:""}.fa-download:before,.rst-content code.download span:first-child:before,.rst-content tt.download span:first-child:before{content:""}.fa-arrow-circle-o-down:before{content:""}.fa-arrow-circle-o-up:before{content:""}.fa-inbox:before{content:""}.fa-play-circle-o:before{content:""}.fa-repeat:before,.fa-rotate-right:before{content:""}.fa-refresh:before{content:""}.fa-list-alt:before{content:""}.fa-lock:before{content:""}.fa-flag:before{content:""}.fa-headphones:before{content:""}.fa-volume-off:before{content:""}.fa-volume-down:before{content:""}.fa-volume-up:before{content:""}.fa-qrcode:before{content:""}.fa-barcode:before{content:""}.fa-tag:before{content:""}.fa-tags:before{content:""}.fa-book:before,.icon-book:before{content:""}.fa-bookmark:before{content:""}.fa-print:before{content:""}.fa-camera:before{content:""}.fa-font:before{content:""}.fa-bold:before{content:""}.fa-italic:before{content:""}.fa-text-height:before{content:""}.fa-text-width:before{content:""}.fa-align-left:before{content:""}.fa-align-center:before{content:""}.fa-align-right:before{content:""}.fa-align-justify:before{content:""}.fa-list:before{content:""}.fa-dedent:before,.fa-outdent:before{content:""}.fa-indent:before{content:""}.fa-video-camera:before{content:""}.fa-image:before,.fa-photo:before,.fa-picture-o:before{content:""}.fa-pencil:before{content:""}.fa-map-marker:before{content:""}.fa-adjust:before{content:""}.fa-tint:before{content:""}.fa-edit:before,.fa-pencil-square-o:before{content:""}.fa-share-square-o:before{content:""}.fa-check-square-o:before{content:""}.fa-arrows:before{content:""}.fa-step-backward:before{content:""}.fa-fast-backward:before{content:""}.fa-backward:before{content:""}.fa-play:before{content:""}.fa-pause:before{content:""}.fa-stop:before{content:""}.fa-forward:before{content:""}.fa-fast-forward:before{content:""}.fa-step-forward:before{content:""}.fa-eject:before{content:""}.fa-chevron-left:before{content:""}.fa-chevron-right:before{content:""}.fa-plus-circle:before{content:""}.fa-minus-circle:before{content:""}.fa-times-circle:before,.wy-inline-validate.wy-inline-validate-danger .wy-input-context:before{content:""}.fa-check-circle:before,.wy-inline-validate.wy-inline-validate-success .wy-input-context:before{content:""}.fa-question-circle:before{content:""}.fa-info-circle:before{content:""}.fa-crosshairs:before{content:""}.fa-times-circle-o:before{content:""}.fa-check-circle-o:before{content:""}.fa-ban:before{content:""}.fa-arrow-left:before{content:""}.fa-arrow-right:before{content:""}.fa-arrow-up:before{content:""}.fa-arrow-down:before{content:""}.fa-mail-forward:before,.fa-share:before{content:""}.fa-expand:before{content:""}.fa-compress:before{content:""}.fa-plus:before{content:""}.fa-minus:before{content:""}.fa-asterisk:before{content:""}.fa-exclamation-circle:before,.rst-content .admonition-title:before,.wy-inline-validate.wy-inline-validate-info .wy-input-context:before,.wy-inline-validate.wy-inline-validate-warning .wy-input-context:before{content:""}.fa-gift:before{content:""}.fa-leaf:before{content:""}.fa-fire:before,.icon-fire:before{content:""}.fa-eye:before{content:""}.fa-eye-slash:before{content:""}.fa-exclamation-triangle:before,.fa-warning:before{content:""}.fa-plane:before{content:""}.fa-calendar:before{content:""}.fa-random:before{content:""}.fa-comment:before{content:""}.fa-magnet:before{content:""}.fa-chevron-up:before{content:""}.fa-chevron-down:before{content:""}.fa-retweet:before{content:""}.fa-shopping-cart:before{content:""}.fa-folder:before{content:""}.fa-folder-open:before{content:""}.fa-arrows-v:before{content:""}.fa-arrows-h:before{content:""}.fa-bar-chart-o:before,.fa-bar-chart:before{content:""}.fa-twitter-square:before{content:""}.fa-facebook-square:before{content:""}.fa-camera-retro:before{content:""}.fa-key:before{content:""}.fa-cogs:before,.fa-gears:before{content:""}.fa-comments:before{content:""}.fa-thumbs-o-up:before{content:""}.fa-thumbs-o-down:before{content:""}.fa-star-half:before{content:""}.fa-heart-o:before{content:""}.fa-sign-out:before{content:""}.fa-linkedin-square:before{content:""}.fa-thumb-tack:before{content:""}.fa-external-link:before{content:""}.fa-sign-in:before{content:""}.fa-trophy:before{content:""}.fa-github-square:before{content:""}.fa-upload:before{content:""}.fa-lemon-o:before{content:""}.fa-phone:before{content:""}.fa-square-o:before{content:""}.fa-bookmark-o:before{content:""}.fa-phone-square:before{content:""}.fa-twitter:before{content:""}.fa-facebook-f:before,.fa-facebook:before{content:""}.fa-github:before,.icon-github:before{content:""}.fa-unlock:before{content:""}.fa-credit-card:before{content:""}.fa-feed:before,.fa-rss:before{content:""}.fa-hdd-o:before{content:""}.fa-bullhorn:before{content:""}.fa-bell:before{content:""}.fa-certificate:before{content:""}.fa-hand-o-right:before{content:""}.fa-hand-o-left:before{content:""}.fa-hand-o-up:before{content:""}.fa-hand-o-down:before{content:""}.fa-arrow-circle-left:before,.icon-circle-arrow-left:before{content:""}.fa-arrow-circle-right:before,.icon-circle-arrow-right:before{content:""}.fa-arrow-circle-up:before{content:""}.fa-arrow-circle-down:before{content:""}.fa-globe:before{content:""}.fa-wrench:before{content:""}.fa-tasks:before{content:""}.fa-filter:before{content:""}.fa-briefcase:before{content:""}.fa-arrows-alt:before{content:""}.fa-group:before,.fa-users:before{content:""}.fa-chain:before,.fa-link:before,.icon-link:before{content:""}.fa-cloud:before{content:""}.fa-flask:before{content:""}.fa-cut:before,.fa-scissors:before{content:""}.fa-copy:before,.fa-files-o:before{content:""}.fa-paperclip:before{content:""}.fa-floppy-o:before,.fa-save:before{content:""}.fa-square:before{content:""}.fa-bars:before,.fa-navicon:before,.fa-reorder:before{content:""}.fa-list-ul:before{content:""}.fa-list-ol:before{content:""}.fa-strikethrough:before{content:""}.fa-underline:before{content:""}.fa-table:before{content:""}.fa-magic:before{content:""}.fa-truck:before{content:""}.fa-pinterest:before{content:""}.fa-pinterest-square:before{content:""}.fa-google-plus-square:before{content:""}.fa-google-plus:before{content:""}.fa-money:before{content:""}.fa-caret-down:before,.icon-caret-down:before,.wy-dropdown .caret:before{content:""}.fa-caret-up:before{content:""}.fa-caret-left:before{content:""}.fa-caret-right:before{content:""}.fa-columns:before{content:""}.fa-sort:before,.fa-unsorted:before{content:""}.fa-sort-desc:before,.fa-sort-down:before{content:""}.fa-sort-asc:before,.fa-sort-up:before{content:""}.fa-envelope:before{content:""}.fa-linkedin:before{content:""}.fa-rotate-left:before,.fa-undo:before{content:""}.fa-gavel:before,.fa-legal:before{content:""}.fa-dashboard:before,.fa-tachometer:before{content:""}.fa-comment-o:before{content:""}.fa-comments-o:before{content:""}.fa-bolt:before,.fa-flash:before{content:""}.fa-sitemap:before{content:""}.fa-umbrella:before{content:""}.fa-clipboard:before,.fa-paste:before{content:""}.fa-lightbulb-o:before{content:""}.fa-exchange:before{content:""}.fa-cloud-download:before{content:""}.fa-cloud-upload:before{content:""}.fa-user-md:before{content:""}.fa-stethoscope:before{content:""}.fa-suitcase:before{content:""}.fa-bell-o:before{content:""}.fa-coffee:before{content:""}.fa-cutlery:before{content:""}.fa-file-text-o:before{content:""}.fa-building-o:before{content:""}.fa-hospital-o:before{content:""}.fa-ambulance:before{content:""}.fa-medkit:before{content:""}.fa-fighter-jet:before{content:""}.fa-beer:before{content:""}.fa-h-square:before{content:""}.fa-plus-square:before{content:""}.fa-angle-double-left:before{content:""}.fa-angle-double-right:before{content:""}.fa-angle-double-up:before{content:""}.fa-angle-double-down:before{content:""}.fa-angle-left:before{content:""}.fa-angle-right:before{content:""}.fa-angle-up:before{content:""}.fa-angle-down:before{content:""}.fa-desktop:before{content:""}.fa-laptop:before{content:""}.fa-tablet:before{content:""}.fa-mobile-phone:before,.fa-mobile:before{content:""}.fa-circle-o:before{content:""}.fa-quote-left:before{content:""}.fa-quote-right:before{content:""}.fa-spinner:before{content:""}.fa-circle:before{content:""}.fa-mail-reply:before,.fa-reply:before{content:""}.fa-github-alt:before{content:""}.fa-folder-o:before{content:""}.fa-folder-open-o:before{content:""}.fa-smile-o:before{content:""}.fa-frown-o:before{content:""}.fa-meh-o:before{content:""}.fa-gamepad:before{content:""}.fa-keyboard-o:before{content:""}.fa-flag-o:before{content:""}.fa-flag-checkered:before{content:""}.fa-terminal:before{content:""}.fa-code:before{content:""}.fa-mail-reply-all:before,.fa-reply-all:before{content:""}.fa-star-half-empty:before,.fa-star-half-full:before,.fa-star-half-o:before{content:""}.fa-location-arrow:before{content:""}.fa-crop:before{content:""}.fa-code-fork:before{content:""}.fa-chain-broken:before,.fa-unlink:before{content:""}.fa-question:before{content:""}.fa-info:before{content:""}.fa-exclamation:before{content:""}.fa-superscript:before{content:""}.fa-subscript:before{content:""}.fa-eraser:before{content:""}.fa-puzzle-piece:before{content:""}.fa-microphone:before{content:""}.fa-microphone-slash:before{content:""}.fa-shield:before{content:""}.fa-calendar-o:before{content:""}.fa-fire-extinguisher:before{content:""}.fa-rocket:before{content:""}.fa-maxcdn:before{content:""}.fa-chevron-circle-left:before{content:""}.fa-chevron-circle-right:before{content:""}.fa-chevron-circle-up:before{content:""}.fa-chevron-circle-down:before{content:""}.fa-html5:before{content:""}.fa-css3:before{content:""}.fa-anchor:before{content:""}.fa-unlock-alt:before{content:""}.fa-bullseye:before{content:""}.fa-ellipsis-h:before{content:""}.fa-ellipsis-v:before{content:""}.fa-rss-square:before{content:""}.fa-play-circle:before{content:""}.fa-ticket:before{content:""}.fa-minus-square:before{content:""}.fa-minus-square-o:before,.wy-menu-vertical li.current>a button.toctree-expand:before,.wy-menu-vertical li.on a button.toctree-expand:before{content:""}.fa-level-up:before{content:""}.fa-level-down:before{content:""}.fa-check-square:before{content:""}.fa-pencil-square:before{content:""}.fa-external-link-square:before{content:""}.fa-share-square:before{content:""}.fa-compass:before{content:""}.fa-caret-square-o-down:before,.fa-toggle-down:before{content:""}.fa-caret-square-o-up:before,.fa-toggle-up:before{content:""}.fa-caret-square-o-right:before,.fa-toggle-right:before{content:""}.fa-eur:before,.fa-euro:before{content:""}.fa-gbp:before{content:""}.fa-dollar:before,.fa-usd:before{content:""}.fa-inr:before,.fa-rupee:before{content:""}.fa-cny:before,.fa-jpy:before,.fa-rmb:before,.fa-yen:before{content:""}.fa-rouble:before,.fa-rub:before,.fa-ruble:before{content:""}.fa-krw:before,.fa-won:before{content:""}.fa-bitcoin:before,.fa-btc:before{content:""}.fa-file:before{content:""}.fa-file-text:before{content:""}.fa-sort-alpha-asc:before{content:""}.fa-sort-alpha-desc:before{content:""}.fa-sort-amount-asc:before{content:""}.fa-sort-amount-desc:before{content:""}.fa-sort-numeric-asc:before{content:""}.fa-sort-numeric-desc:before{content:""}.fa-thumbs-up:before{content:""}.fa-thumbs-down:before{content:""}.fa-youtube-square:before{content:""}.fa-youtube:before{content:""}.fa-xing:before{content:""}.fa-xing-square:before{content:""}.fa-youtube-play:before{content:""}.fa-dropbox:before{content:""}.fa-stack-overflow:before{content:""}.fa-instagram:before{content:""}.fa-flickr:before{content:""}.fa-adn:before{content:""}.fa-bitbucket:before,.icon-bitbucket:before{content:""}.fa-bitbucket-square:before{content:""}.fa-tumblr:before{content:""}.fa-tumblr-square:before{content:""}.fa-long-arrow-down:before{content:""}.fa-long-arrow-up:before{content:""}.fa-long-arrow-left:before{content:""}.fa-long-arrow-right:before{content:""}.fa-apple:before{content:""}.fa-windows:before{content:""}.fa-android:before{content:""}.fa-linux:before{content:""}.fa-dribbble:before{content:""}.fa-skype:before{content:""}.fa-foursquare:before{content:""}.fa-trello:before{content:""}.fa-female:before{content:""}.fa-male:before{content:""}.fa-gittip:before,.fa-gratipay:before{content:""}.fa-sun-o:before{content:""}.fa-moon-o:before{content:""}.fa-archive:before{content:""}.fa-bug:before{content:""}.fa-vk:before{content:""}.fa-weibo:before{content:""}.fa-renren:before{content:""}.fa-pagelines:before{content:""}.fa-stack-exchange:before{content:""}.fa-arrow-circle-o-right:before{content:""}.fa-arrow-circle-o-left:before{content:""}.fa-caret-square-o-left:before,.fa-toggle-left:before{content:""}.fa-dot-circle-o:before{content:""}.fa-wheelchair:before{content:""}.fa-vimeo-square:before{content:""}.fa-try:before,.fa-turkish-lira:before{content:""}.fa-plus-square-o:before,.wy-menu-vertical li button.toctree-expand:before{content:""}.fa-space-shuttle:before{content:""}.fa-slack:before{content:""}.fa-envelope-square:before{content:""}.fa-wordpress:before{content:""}.fa-openid:before{content:""}.fa-bank:before,.fa-institution:before,.fa-university:before{content:""}.fa-graduation-cap:before,.fa-mortar-board:before{content:""}.fa-yahoo:before{content:""}.fa-google:before{content:""}.fa-reddit:before{content:""}.fa-reddit-square:before{content:""}.fa-stumbleupon-circle:before{content:""}.fa-stumbleupon:before{content:""}.fa-delicious:before{content:""}.fa-digg:before{content:""}.fa-pied-piper-pp:before{content:""}.fa-pied-piper-alt:before{content:""}.fa-drupal:before{content:""}.fa-joomla:before{content:""}.fa-language:before{content:""}.fa-fax:before{content:""}.fa-building:before{content:""}.fa-child:before{content:""}.fa-paw:before{content:""}.fa-spoon:before{content:""}.fa-cube:before{content:""}.fa-cubes:before{content:""}.fa-behance:before{content:""}.fa-behance-square:before{content:""}.fa-steam:before{content:""}.fa-steam-square:before{content:""}.fa-recycle:before{content:""}.fa-automobile:before,.fa-car:before{content:""}.fa-cab:before,.fa-taxi:before{content:""}.fa-tree:before{content:""}.fa-spotify:before{content:""}.fa-deviantart:before{content:""}.fa-soundcloud:before{content:""}.fa-database:before{content:""}.fa-file-pdf-o:before{content:""}.fa-file-word-o:before{content:""}.fa-file-excel-o:before{content:""}.fa-file-powerpoint-o:before{content:""}.fa-file-image-o:before,.fa-file-photo-o:before,.fa-file-picture-o:before{content:""}.fa-file-archive-o:before,.fa-file-zip-o:before{content:""}.fa-file-audio-o:before,.fa-file-sound-o:before{content:""}.fa-file-movie-o:before,.fa-file-video-o:before{content:""}.fa-file-code-o:before{content:""}.fa-vine:before{content:""}.fa-codepen:before{content:""}.fa-jsfiddle:before{content:""}.fa-life-bouy:before,.fa-life-buoy:before,.fa-life-ring:before,.fa-life-saver:before,.fa-support:before{content:""}.fa-circle-o-notch:before{content:""}.fa-ra:before,.fa-rebel:before,.fa-resistance:before{content:""}.fa-empire:before,.fa-ge:before{content:""}.fa-git-square:before{content:""}.fa-git:before{content:""}.fa-hacker-news:before,.fa-y-combinator-square:before,.fa-yc-square:before{content:""}.fa-tencent-weibo:before{content:""}.fa-qq:before{content:""}.fa-wechat:before,.fa-weixin:before{content:""}.fa-paper-plane:before,.fa-send:before{content:""}.fa-paper-plane-o:before,.fa-send-o:before{content:""}.fa-history:before{content:""}.fa-circle-thin:before{content:""}.fa-header:before{content:""}.fa-paragraph:before{content:""}.fa-sliders:before{content:""}.fa-share-alt:before{content:""}.fa-share-alt-square:before{content:""}.fa-bomb:before{content:""}.fa-futbol-o:before,.fa-soccer-ball-o:before{content:""}.fa-tty:before{content:""}.fa-binoculars:before{content:""}.fa-plug:before{content:""}.fa-slideshare:before{content:""}.fa-twitch:before{content:""}.fa-yelp:before{content:""}.fa-newspaper-o:before{content:""}.fa-wifi:before{content:""}.fa-calculator:before{content:""}.fa-paypal:before{content:""}.fa-google-wallet:before{content:""}.fa-cc-visa:before{content:""}.fa-cc-mastercard:before{content:""}.fa-cc-discover:before{content:""}.fa-cc-amex:before{content:""}.fa-cc-paypal:before{content:""}.fa-cc-stripe:before{content:""}.fa-bell-slash:before{content:""}.fa-bell-slash-o:before{content:""}.fa-trash:before{content:""}.fa-copyright:before{content:""}.fa-at:before{content:""}.fa-eyedropper:before{content:""}.fa-paint-brush:before{content:""}.fa-birthday-cake:before{content:""}.fa-area-chart:before{content:""}.fa-pie-chart:before{content:""}.fa-line-chart:before{content:""}.fa-lastfm:before{content:""}.fa-lastfm-square:before{content:""}.fa-toggle-off:before{content:""}.fa-toggle-on:before{content:""}.fa-bicycle:before{content:""}.fa-bus:before{content:""}.fa-ioxhost:before{content:""}.fa-angellist:before{content:""}.fa-cc:before{content:""}.fa-ils:before,.fa-shekel:before,.fa-sheqel:before{content:""}.fa-meanpath:before{content:""}.fa-buysellads:before{content:""}.fa-connectdevelop:before{content:""}.fa-dashcube:before{content:""}.fa-forumbee:before{content:""}.fa-leanpub:before{content:""}.fa-sellsy:before{content:""}.fa-shirtsinbulk:before{content:""}.fa-simplybuilt:before{content:""}.fa-skyatlas:before{content:""}.fa-cart-plus:before{content:""}.fa-cart-arrow-down:before{content:""}.fa-diamond:before{content:""}.fa-ship:before{content:""}.fa-user-secret:before{content:""}.fa-motorcycle:before{content:""}.fa-street-view:before{content:""}.fa-heartbeat:before{content:""}.fa-venus:before{content:""}.fa-mars:before{content:""}.fa-mercury:before{content:""}.fa-intersex:before,.fa-transgender:before{content:""}.fa-transgender-alt:before{content:""}.fa-venus-double:before{content:""}.fa-mars-double:before{content:""}.fa-venus-mars:before{content:""}.fa-mars-stroke:before{content:""}.fa-mars-stroke-v:before{content:""}.fa-mars-stroke-h:before{content:""}.fa-neuter:before{content:""}.fa-genderless:before{content:""}.fa-facebook-official:before{content:""}.fa-pinterest-p:before{content:""}.fa-whatsapp:before{content:""}.fa-server:before{content:""}.fa-user-plus:before{content:""}.fa-user-times:before{content:""}.fa-bed:before,.fa-hotel:before{content:""}.fa-viacoin:before{content:""}.fa-train:before{content:""}.fa-subway:before{content:""}.fa-medium:before{content:""}.fa-y-combinator:before,.fa-yc:before{content:""}.fa-optin-monster:before{content:""}.fa-opencart:before{content:""}.fa-expeditedssl:before{content:""}.fa-battery-4:before,.fa-battery-full:before,.fa-battery:before{content:""}.fa-battery-3:before,.fa-battery-three-quarters:before{content:""}.fa-battery-2:before,.fa-battery-half:before{content:""}.fa-battery-1:before,.fa-battery-quarter:before{content:""}.fa-battery-0:before,.fa-battery-empty:before{content:""}.fa-mouse-pointer:before{content:""}.fa-i-cursor:before{content:""}.fa-object-group:before{content:""}.fa-object-ungroup:before{content:""}.fa-sticky-note:before{content:""}.fa-sticky-note-o:before{content:""}.fa-cc-jcb:before{content:""}.fa-cc-diners-club:before{content:""}.fa-clone:before{content:""}.fa-balance-scale:before{content:""}.fa-hourglass-o:before{content:""}.fa-hourglass-1:before,.fa-hourglass-start:before{content:""}.fa-hourglass-2:before,.fa-hourglass-half:before{content:""}.fa-hourglass-3:before,.fa-hourglass-end:before{content:""}.fa-hourglass:before{content:""}.fa-hand-grab-o:before,.fa-hand-rock-o:before{content:""}.fa-hand-paper-o:before,.fa-hand-stop-o:before{content:""}.fa-hand-scissors-o:before{content:""}.fa-hand-lizard-o:before{content:""}.fa-hand-spock-o:before{content:""}.fa-hand-pointer-o:before{content:""}.fa-hand-peace-o:before{content:""}.fa-trademark:before{content:""}.fa-registered:before{content:""}.fa-creative-commons:before{content:""}.fa-gg:before{content:""}.fa-gg-circle:before{content:""}.fa-tripadvisor:before{content:""}.fa-odnoklassniki:before{content:""}.fa-odnoklassniki-square:before{content:""}.fa-get-pocket:before{content:""}.fa-wikipedia-w:before{content:""}.fa-safari:before{content:""}.fa-chrome:before{content:""}.fa-firefox:before{content:""}.fa-opera:before{content:""}.fa-internet-explorer:before{content:""}.fa-television:before,.fa-tv:before{content:""}.fa-contao:before{content:""}.fa-500px:before{content:""}.fa-amazon:before{content:""}.fa-calendar-plus-o:before{content:""}.fa-calendar-minus-o:before{content:""}.fa-calendar-times-o:before{content:""}.fa-calendar-check-o:before{content:""}.fa-industry:before{content:""}.fa-map-pin:before{content:""}.fa-map-signs:before{content:""}.fa-map-o:before{content:""}.fa-map:before{content:""}.fa-commenting:before{content:""}.fa-commenting-o:before{content:""}.fa-houzz:before{content:""}.fa-vimeo:before{content:""}.fa-black-tie:before{content:""}.fa-fonticons:before{content:""}.fa-reddit-alien:before{content:""}.fa-edge:before{content:""}.fa-credit-card-alt:before{content:""}.fa-codiepie:before{content:""}.fa-modx:before{content:""}.fa-fort-awesome:before{content:""}.fa-usb:before{content:""}.fa-product-hunt:before{content:""}.fa-mixcloud:before{content:""}.fa-scribd:before{content:""}.fa-pause-circle:before{content:""}.fa-pause-circle-o:before{content:""}.fa-stop-circle:before{content:""}.fa-stop-circle-o:before{content:""}.fa-shopping-bag:before{content:""}.fa-shopping-basket:before{content:""}.fa-hashtag:before{content:""}.fa-bluetooth:before{content:""}.fa-bluetooth-b:before{content:""}.fa-percent:before{content:""}.fa-gitlab:before,.icon-gitlab:before{content:""}.fa-wpbeginner:before{content:""}.fa-wpforms:before{content:""}.fa-envira:before{content:""}.fa-universal-access:before{content:""}.fa-wheelchair-alt:before{content:""}.fa-question-circle-o:before{content:""}.fa-blind:before{content:""}.fa-audio-description:before{content:""}.fa-volume-control-phone:before{content:""}.fa-braille:before{content:""}.fa-assistive-listening-systems:before{content:""}.fa-american-sign-language-interpreting:before,.fa-asl-interpreting:before{content:""}.fa-deaf:before,.fa-deafness:before,.fa-hard-of-hearing:before{content:""}.fa-glide:before{content:""}.fa-glide-g:before{content:""}.fa-sign-language:before,.fa-signing:before{content:""}.fa-low-vision:before{content:""}.fa-viadeo:before{content:""}.fa-viadeo-square:before{content:""}.fa-snapchat:before{content:""}.fa-snapchat-ghost:before{content:""}.fa-snapchat-square:before{content:""}.fa-pied-piper:before{content:""}.fa-first-order:before{content:""}.fa-yoast:before{content:""}.fa-themeisle:before{content:""}.fa-google-plus-circle:before,.fa-google-plus-official:before{content:""}.fa-fa:before,.fa-font-awesome:before{content:""}.fa-handshake-o:before{content:""}.fa-envelope-open:before{content:""}.fa-envelope-open-o:before{content:""}.fa-linode:before{content:""}.fa-address-book:before{content:""}.fa-address-book-o:before{content:""}.fa-address-card:before,.fa-vcard:before{content:""}.fa-address-card-o:before,.fa-vcard-o:before{content:""}.fa-user-circle:before{content:""}.fa-user-circle-o:before{content:""}.fa-user-o:before{content:""}.fa-id-badge:before{content:""}.fa-drivers-license:before,.fa-id-card:before{content:""}.fa-drivers-license-o:before,.fa-id-card-o:before{content:""}.fa-quora:before{content:""}.fa-free-code-camp:before{content:""}.fa-telegram:before{content:""}.fa-thermometer-4:before,.fa-thermometer-full:before,.fa-thermometer:before{content:""}.fa-thermometer-3:before,.fa-thermometer-three-quarters:before{content:""}.fa-thermometer-2:before,.fa-thermometer-half:before{content:""}.fa-thermometer-1:before,.fa-thermometer-quarter:before{content:""}.fa-thermometer-0:before,.fa-thermometer-empty:before{content:""}.fa-shower:before{content:""}.fa-bath:before,.fa-bathtub:before,.fa-s15:before{content:""}.fa-podcast:before{content:""}.fa-window-maximize:before{content:""}.fa-window-minimize:before{content:""}.fa-window-restore:before{content:""}.fa-times-rectangle:before,.fa-window-close:before{content:""}.fa-times-rectangle-o:before,.fa-window-close-o:before{content:""}.fa-bandcamp:before{content:""}.fa-grav:before{content:""}.fa-etsy:before{content:""}.fa-imdb:before{content:""}.fa-ravelry:before{content:""}.fa-eercast:before{content:""}.fa-microchip:before{content:""}.fa-snowflake-o:before{content:""}.fa-superpowers:before{content:""}.fa-wpexplorer:before{content:""}.fa-meetup:before{content:""}.sr-only{position:absolute;width:1px;height:1px;padding:0;margin:-1px;overflow:hidden;clip:rect(0,0,0,0);border:0}.sr-only-focusable:active,.sr-only-focusable:focus{position:static;width:auto;height:auto;margin:0;overflow:visible;clip:auto}.fa,.icon,.rst-content .admonition-title,.rst-content .code-block-caption .headerlink,.rst-content .eqno .headerlink,.rst-content code.download span:first-child,.rst-content dl dt .headerlink,.rst-content h1 .headerlink,.rst-content h2 .headerlink,.rst-content h3 .headerlink,.rst-content h4 .headerlink,.rst-content h5 .headerlink,.rst-content h6 .headerlink,.rst-content p.caption .headerlink,.rst-content p .headerlink,.rst-content table>caption .headerlink,.rst-content tt.download span:first-child,.wy-dropdown .caret,.wy-inline-validate.wy-inline-validate-danger .wy-input-context,.wy-inline-validate.wy-inline-validate-info .wy-input-context,.wy-inline-validate.wy-inline-validate-success .wy-input-context,.wy-inline-validate.wy-inline-validate-warning .wy-input-context,.wy-menu-vertical li.current>a button.toctree-expand,.wy-menu-vertical li.on a button.toctree-expand,.wy-menu-vertical li button.toctree-expand{font-family:inherit}.fa:before,.icon:before,.rst-content .admonition-title:before,.rst-content .code-block-caption .headerlink:before,.rst-content .eqno .headerlink:before,.rst-content code.download span:first-child:before,.rst-content dl dt .headerlink:before,.rst-content h1 .headerlink:before,.rst-content h2 .headerlink:before,.rst-content h3 .headerlink:before,.rst-content h4 .headerlink:before,.rst-content h5 .headerlink:before,.rst-content h6 .headerlink:before,.rst-content p.caption .headerlink:before,.rst-content p .headerlink:before,.rst-content table>caption .headerlink:before,.rst-content tt.download span:first-child:before,.wy-dropdown .caret:before,.wy-inline-validate.wy-inline-validate-danger .wy-input-context:before,.wy-inline-validate.wy-inline-validate-info .wy-input-context:before,.wy-inline-validate.wy-inline-validate-success .wy-input-context:before,.wy-inline-validate.wy-inline-validate-warning .wy-input-context:before,.wy-menu-vertical li.current>a button.toctree-expand:before,.wy-menu-vertical li.on a button.toctree-expand:before,.wy-menu-vertical li button.toctree-expand:before{font-family:FontAwesome;display:inline-block;font-style:normal;font-weight:400;line-height:1;text-decoration:inherit}.rst-content .code-block-caption a .headerlink,.rst-content .eqno a .headerlink,.rst-content a .admonition-title,.rst-content code.download a span:first-child,.rst-content dl dt a .headerlink,.rst-content h1 a .headerlink,.rst-content h2 a .headerlink,.rst-content h3 a .headerlink,.rst-content h4 a .headerlink,.rst-content h5 a .headerlink,.rst-content h6 a .headerlink,.rst-content p.caption a .headerlink,.rst-content p a .headerlink,.rst-content table>caption a .headerlink,.rst-content tt.download a span:first-child,.wy-menu-vertical li.current>a button.toctree-expand,.wy-menu-vertical li.on a button.toctree-expand,.wy-menu-vertical li a button.toctree-expand,a .fa,a .icon,a .rst-content .admonition-title,a .rst-content .code-block-caption .headerlink,a .rst-content .eqno .headerlink,a .rst-content code.download span:first-child,a .rst-content dl dt .headerlink,a .rst-content h1 .headerlink,a .rst-content h2 .headerlink,a .rst-content h3 .headerlink,a .rst-content h4 .headerlink,a .rst-content h5 .headerlink,a .rst-content h6 .headerlink,a .rst-content p.caption .headerlink,a .rst-content p .headerlink,a .rst-content table>caption .headerlink,a .rst-content tt.download span:first-child,a .wy-menu-vertical li button.toctree-expand{display:inline-block;text-decoration:inherit}.btn .fa,.btn .icon,.btn .rst-content .admonition-title,.btn .rst-content .code-block-caption .headerlink,.btn .rst-content .eqno .headerlink,.btn .rst-content code.download span:first-child,.btn .rst-content dl dt .headerlink,.btn .rst-content h1 .headerlink,.btn .rst-content h2 .headerlink,.btn .rst-content h3 .headerlink,.btn .rst-content h4 .headerlink,.btn .rst-content h5 .headerlink,.btn .rst-content h6 .headerlink,.btn .rst-content p .headerlink,.btn .rst-content table>caption .headerlink,.btn .rst-content tt.download span:first-child,.btn .wy-menu-vertical li.current>a button.toctree-expand,.btn .wy-menu-vertical li.on a button.toctree-expand,.btn .wy-menu-vertical li button.toctree-expand,.nav .fa,.nav .icon,.nav .rst-content .admonition-title,.nav .rst-content .code-block-caption .headerlink,.nav .rst-content .eqno .headerlink,.nav .rst-content code.download span:first-child,.nav .rst-content dl dt .headerlink,.nav .rst-content h1 .headerlink,.nav .rst-content h2 .headerlink,.nav .rst-content h3 .headerlink,.nav .rst-content h4 .headerlink,.nav .rst-content h5 .headerlink,.nav .rst-content h6 .headerlink,.nav .rst-content p .headerlink,.nav .rst-content table>caption .headerlink,.nav .rst-content tt.download span:first-child,.nav .wy-menu-vertical li.current>a button.toctree-expand,.nav .wy-menu-vertical li.on a button.toctree-expand,.nav .wy-menu-vertical li button.toctree-expand,.rst-content .btn .admonition-title,.rst-content .code-block-caption .btn .headerlink,.rst-content .code-block-caption .nav .headerlink,.rst-content .eqno .btn .headerlink,.rst-content .eqno .nav .headerlink,.rst-content .nav .admonition-title,.rst-content code.download .btn span:first-child,.rst-content code.download .nav span:first-child,.rst-content dl dt .btn .headerlink,.rst-content dl dt .nav .headerlink,.rst-content h1 .btn .headerlink,.rst-content h1 .nav .headerlink,.rst-content h2 .btn .headerlink,.rst-content h2 .nav .headerlink,.rst-content h3 .btn .headerlink,.rst-content h3 .nav .headerlink,.rst-content h4 .btn .headerlink,.rst-content h4 .nav .headerlink,.rst-content h5 .btn .headerlink,.rst-content h5 .nav .headerlink,.rst-content h6 .btn .headerlink,.rst-content h6 .nav .headerlink,.rst-content p .btn .headerlink,.rst-content p .nav .headerlink,.rst-content table>caption .btn .headerlink,.rst-content table>caption .nav .headerlink,.rst-content tt.download .btn span:first-child,.rst-content tt.download .nav span:first-child,.wy-menu-vertical li .btn button.toctree-expand,.wy-menu-vertical li.current>a .btn button.toctree-expand,.wy-menu-vertical li.current>a .nav button.toctree-expand,.wy-menu-vertical li .nav button.toctree-expand,.wy-menu-vertical li.on a .btn button.toctree-expand,.wy-menu-vertical li.on a .nav button.toctree-expand{display:inline}.btn .fa-large.icon,.btn .fa.fa-large,.btn .rst-content .code-block-caption .fa-large.headerlink,.btn .rst-content .eqno .fa-large.headerlink,.btn .rst-content .fa-large.admonition-title,.btn .rst-content code.download span.fa-large:first-child,.btn .rst-content dl dt .fa-large.headerlink,.btn .rst-content h1 .fa-large.headerlink,.btn .rst-content h2 .fa-large.headerlink,.btn .rst-content h3 .fa-large.headerlink,.btn .rst-content h4 .fa-large.headerlink,.btn .rst-content h5 .fa-large.headerlink,.btn .rst-content h6 .fa-large.headerlink,.btn .rst-content p .fa-large.headerlink,.btn .rst-content table>caption .fa-large.headerlink,.btn .rst-content tt.download span.fa-large:first-child,.btn .wy-menu-vertical li button.fa-large.toctree-expand,.nav .fa-large.icon,.nav .fa.fa-large,.nav .rst-content .code-block-caption .fa-large.headerlink,.nav .rst-content .eqno .fa-large.headerlink,.nav .rst-content .fa-large.admonition-title,.nav .rst-content code.download span.fa-large:first-child,.nav .rst-content dl dt .fa-large.headerlink,.nav .rst-content h1 .fa-large.headerlink,.nav .rst-content h2 .fa-large.headerlink,.nav .rst-content h3 .fa-large.headerlink,.nav .rst-content h4 .fa-large.headerlink,.nav .rst-content h5 .fa-large.headerlink,.nav .rst-content h6 .fa-large.headerlink,.nav .rst-content p .fa-large.headerlink,.nav .rst-content table>caption .fa-large.headerlink,.nav .rst-content tt.download span.fa-large:first-child,.nav .wy-menu-vertical li button.fa-large.toctree-expand,.rst-content .btn .fa-large.admonition-title,.rst-content .code-block-caption .btn .fa-large.headerlink,.rst-content .code-block-caption .nav .fa-large.headerlink,.rst-content .eqno .btn .fa-large.headerlink,.rst-content .eqno .nav .fa-large.headerlink,.rst-content .nav .fa-large.admonition-title,.rst-content code.download .btn span.fa-large:first-child,.rst-content code.download .nav span.fa-large:first-child,.rst-content dl dt .btn .fa-large.headerlink,.rst-content dl dt .nav .fa-large.headerlink,.rst-content h1 .btn .fa-large.headerlink,.rst-content h1 .nav .fa-large.headerlink,.rst-content h2 .btn .fa-large.headerlink,.rst-content h2 .nav .fa-large.headerlink,.rst-content h3 .btn .fa-large.headerlink,.rst-content h3 .nav .fa-large.headerlink,.rst-content h4 .btn .fa-large.headerlink,.rst-content h4 .nav .fa-large.headerlink,.rst-content h5 .btn .fa-large.headerlink,.rst-content h5 .nav .fa-large.headerlink,.rst-content h6 .btn .fa-large.headerlink,.rst-content h6 .nav .fa-large.headerlink,.rst-content p .btn .fa-large.headerlink,.rst-content p .nav .fa-large.headerlink,.rst-content table>caption .btn .fa-large.headerlink,.rst-content table>caption .nav .fa-large.headerlink,.rst-content tt.download .btn span.fa-large:first-child,.rst-content tt.download .nav span.fa-large:first-child,.wy-menu-vertical li .btn button.fa-large.toctree-expand,.wy-menu-vertical li .nav button.fa-large.toctree-expand{line-height:.9em}.btn .fa-spin.icon,.btn .fa.fa-spin,.btn .rst-content .code-block-caption .fa-spin.headerlink,.btn .rst-content .eqno .fa-spin.headerlink,.btn .rst-content .fa-spin.admonition-title,.btn .rst-content code.download span.fa-spin:first-child,.btn .rst-content dl dt .fa-spin.headerlink,.btn .rst-content h1 .fa-spin.headerlink,.btn .rst-content h2 .fa-spin.headerlink,.btn .rst-content h3 .fa-spin.headerlink,.btn .rst-content h4 .fa-spin.headerlink,.btn .rst-content h5 .fa-spin.headerlink,.btn .rst-content h6 .fa-spin.headerlink,.btn .rst-content p .fa-spin.headerlink,.btn .rst-content table>caption .fa-spin.headerlink,.btn .rst-content tt.download span.fa-spin:first-child,.btn .wy-menu-vertical li button.fa-spin.toctree-expand,.nav .fa-spin.icon,.nav .fa.fa-spin,.nav .rst-content .code-block-caption .fa-spin.headerlink,.nav .rst-content .eqno .fa-spin.headerlink,.nav .rst-content .fa-spin.admonition-title,.nav .rst-content code.download span.fa-spin:first-child,.nav .rst-content dl dt .fa-spin.headerlink,.nav .rst-content h1 .fa-spin.headerlink,.nav .rst-content h2 .fa-spin.headerlink,.nav .rst-content h3 .fa-spin.headerlink,.nav .rst-content h4 .fa-spin.headerlink,.nav .rst-content h5 .fa-spin.headerlink,.nav .rst-content h6 .fa-spin.headerlink,.nav .rst-content p .fa-spin.headerlink,.nav .rst-content table>caption .fa-spin.headerlink,.nav .rst-content tt.download span.fa-spin:first-child,.nav .wy-menu-vertical li button.fa-spin.toctree-expand,.rst-content .btn .fa-spin.admonition-title,.rst-content .code-block-caption .btn .fa-spin.headerlink,.rst-content .code-block-caption .nav .fa-spin.headerlink,.rst-content .eqno .btn .fa-spin.headerlink,.rst-content .eqno .nav .fa-spin.headerlink,.rst-content .nav .fa-spin.admonition-title,.rst-content code.download .btn span.fa-spin:first-child,.rst-content code.download .nav span.fa-spin:first-child,.rst-content dl dt .btn .fa-spin.headerlink,.rst-content dl dt .nav .fa-spin.headerlink,.rst-content h1 .btn .fa-spin.headerlink,.rst-content h1 .nav .fa-spin.headerlink,.rst-content h2 .btn .fa-spin.headerlink,.rst-content h2 .nav .fa-spin.headerlink,.rst-content h3 .btn .fa-spin.headerlink,.rst-content h3 .nav .fa-spin.headerlink,.rst-content h4 .btn .fa-spin.headerlink,.rst-content h4 .nav .fa-spin.headerlink,.rst-content h5 .btn .fa-spin.headerlink,.rst-content h5 .nav .fa-spin.headerlink,.rst-content h6 .btn .fa-spin.headerlink,.rst-content h6 .nav .fa-spin.headerlink,.rst-content p .btn .fa-spin.headerlink,.rst-content p .nav .fa-spin.headerlink,.rst-content table>caption .btn .fa-spin.headerlink,.rst-content table>caption .nav .fa-spin.headerlink,.rst-content tt.download .btn span.fa-spin:first-child,.rst-content tt.download .nav span.fa-spin:first-child,.wy-menu-vertical li .btn button.fa-spin.toctree-expand,.wy-menu-vertical li .nav button.fa-spin.toctree-expand{display:inline-block}.btn.fa:before,.btn.icon:before,.rst-content .btn.admonition-title:before,.rst-content .code-block-caption .btn.headerlink:before,.rst-content .eqno .btn.headerlink:before,.rst-content code.download span.btn:first-child:before,.rst-content dl dt .btn.headerlink:before,.rst-content h1 .btn.headerlink:before,.rst-content h2 .btn.headerlink:before,.rst-content h3 .btn.headerlink:before,.rst-content h4 .btn.headerlink:before,.rst-content h5 .btn.headerlink:before,.rst-content h6 .btn.headerlink:before,.rst-content p .btn.headerlink:before,.rst-content table>caption .btn.headerlink:before,.rst-content tt.download span.btn:first-child:before,.wy-menu-vertical li button.btn.toctree-expand:before{opacity:.5;-webkit-transition:opacity .05s ease-in;-moz-transition:opacity .05s ease-in;transition:opacity .05s ease-in}.btn.fa:hover:before,.btn.icon:hover:before,.rst-content .btn.admonition-title:hover:before,.rst-content .code-block-caption .btn.headerlink:hover:before,.rst-content .eqno .btn.headerlink:hover:before,.rst-content code.download span.btn:first-child:hover:before,.rst-content dl dt .btn.headerlink:hover:before,.rst-content h1 .btn.headerlink:hover:before,.rst-content h2 .btn.headerlink:hover:before,.rst-content h3 .btn.headerlink:hover:before,.rst-content h4 .btn.headerlink:hover:before,.rst-content h5 .btn.headerlink:hover:before,.rst-content h6 .btn.headerlink:hover:before,.rst-content p .btn.headerlink:hover:before,.rst-content table>caption .btn.headerlink:hover:before,.rst-content tt.download span.btn:first-child:hover:before,.wy-menu-vertical li button.btn.toctree-expand:hover:before{opacity:1}.btn-mini .fa:before,.btn-mini .icon:before,.btn-mini .rst-content .admonition-title:before,.btn-mini .rst-content .code-block-caption .headerlink:before,.btn-mini .rst-content .eqno .headerlink:before,.btn-mini .rst-content code.download span:first-child:before,.btn-mini .rst-content dl dt .headerlink:before,.btn-mini .rst-content h1 .headerlink:before,.btn-mini .rst-content h2 .headerlink:before,.btn-mini .rst-content h3 .headerlink:before,.btn-mini .rst-content h4 .headerlink:before,.btn-mini .rst-content h5 .headerlink:before,.btn-mini .rst-content h6 .headerlink:before,.btn-mini .rst-content p .headerlink:before,.btn-mini .rst-content table>caption .headerlink:before,.btn-mini .rst-content tt.download span:first-child:before,.btn-mini .wy-menu-vertical li button.toctree-expand:before,.rst-content .btn-mini .admonition-title:before,.rst-content .code-block-caption .btn-mini .headerlink:before,.rst-content .eqno .btn-mini .headerlink:before,.rst-content code.download .btn-mini span:first-child:before,.rst-content dl dt .btn-mini .headerlink:before,.rst-content h1 .btn-mini .headerlink:before,.rst-content h2 .btn-mini .headerlink:before,.rst-content h3 .btn-mini .headerlink:before,.rst-content h4 .btn-mini .headerlink:before,.rst-content h5 .btn-mini .headerlink:before,.rst-content h6 .btn-mini .headerlink:before,.rst-content p .btn-mini .headerlink:before,.rst-content table>caption .btn-mini .headerlink:before,.rst-content tt.download .btn-mini span:first-child:before,.wy-menu-vertical li .btn-mini button.toctree-expand:before{font-size:14px;vertical-align:-15%}.rst-content .admonition,.rst-content .admonition-todo,.rst-content .attention,.rst-content .caution,.rst-content .danger,.rst-content .error,.rst-content .hint,.rst-content .important,.rst-content .note,.rst-content .seealso,.rst-content .tip,.rst-content .warning,.wy-alert{padding:12px;line-height:24px;margin-bottom:24px;background:#e7f2fa}.rst-content .admonition-title,.wy-alert-title{font-weight:700;display:block;color:#fff;background:#6ab0de;padding:6px 12px;margin:-12px -12px 12px}.rst-content .danger,.rst-content .error,.rst-content .wy-alert-danger.admonition,.rst-content .wy-alert-danger.admonition-todo,.rst-content .wy-alert-danger.attention,.rst-content .wy-alert-danger.caution,.rst-content .wy-alert-danger.hint,.rst-content .wy-alert-danger.important,.rst-content .wy-alert-danger.note,.rst-content .wy-alert-danger.seealso,.rst-content .wy-alert-danger.tip,.rst-content .wy-alert-danger.warning,.wy-alert.wy-alert-danger{background:#fdf3f2}.rst-content .danger .admonition-title,.rst-content .danger .wy-alert-title,.rst-content .error .admonition-title,.rst-content .error .wy-alert-title,.rst-content .wy-alert-danger.admonition-todo .admonition-title,.rst-content .wy-alert-danger.admonition-todo .wy-alert-title,.rst-content .wy-alert-danger.admonition .admonition-title,.rst-content .wy-alert-danger.admonition .wy-alert-title,.rst-content .wy-alert-danger.attention .admonition-title,.rst-content .wy-alert-danger.attention .wy-alert-title,.rst-content .wy-alert-danger.caution .admonition-title,.rst-content .wy-alert-danger.caution .wy-alert-title,.rst-content .wy-alert-danger.hint .admonition-title,.rst-content .wy-alert-danger.hint .wy-alert-title,.rst-content .wy-alert-danger.important .admonition-title,.rst-content .wy-alert-danger.important .wy-alert-title,.rst-content .wy-alert-danger.note .admonition-title,.rst-content .wy-alert-danger.note .wy-alert-title,.rst-content .wy-alert-danger.seealso .admonition-title,.rst-content .wy-alert-danger.seealso .wy-alert-title,.rst-content .wy-alert-danger.tip .admonition-title,.rst-content .wy-alert-danger.tip .wy-alert-title,.rst-content .wy-alert-danger.warning .admonition-title,.rst-content .wy-alert-danger.warning .wy-alert-title,.rst-content .wy-alert.wy-alert-danger .admonition-title,.wy-alert.wy-alert-danger .rst-content .admonition-title,.wy-alert.wy-alert-danger .wy-alert-title{background:#f29f97}.rst-content .admonition-todo,.rst-content .attention,.rst-content .caution,.rst-content .warning,.rst-content .wy-alert-warning.admonition,.rst-content .wy-alert-warning.danger,.rst-content .wy-alert-warning.error,.rst-content .wy-alert-warning.hint,.rst-content .wy-alert-warning.important,.rst-content .wy-alert-warning.note,.rst-content .wy-alert-warning.seealso,.rst-content .wy-alert-warning.tip,.wy-alert.wy-alert-warning{background:#ffedcc}.rst-content .admonition-todo .admonition-title,.rst-content .admonition-todo .wy-alert-title,.rst-content .attention .admonition-title,.rst-content .attention .wy-alert-title,.rst-content .caution .admonition-title,.rst-content .caution .wy-alert-title,.rst-content .warning .admonition-title,.rst-content .warning .wy-alert-title,.rst-content .wy-alert-warning.admonition .admonition-title,.rst-content .wy-alert-warning.admonition .wy-alert-title,.rst-content .wy-alert-warning.danger .admonition-title,.rst-content .wy-alert-warning.danger .wy-alert-title,.rst-content .wy-alert-warning.error .admonition-title,.rst-content .wy-alert-warning.error .wy-alert-title,.rst-content .wy-alert-warning.hint .admonition-title,.rst-content .wy-alert-warning.hint .wy-alert-title,.rst-content .wy-alert-warning.important .admonition-title,.rst-content .wy-alert-warning.important .wy-alert-title,.rst-content .wy-alert-warning.note .admonition-title,.rst-content .wy-alert-warning.note .wy-alert-title,.rst-content .wy-alert-warning.seealso .admonition-title,.rst-content .wy-alert-warning.seealso .wy-alert-title,.rst-content .wy-alert-warning.tip .admonition-title,.rst-content .wy-alert-warning.tip .wy-alert-title,.rst-content .wy-alert.wy-alert-warning .admonition-title,.wy-alert.wy-alert-warning .rst-content .admonition-title,.wy-alert.wy-alert-warning .wy-alert-title{background:#f0b37e}.rst-content .note,.rst-content .seealso,.rst-content .wy-alert-info.admonition,.rst-content .wy-alert-info.admonition-todo,.rst-content .wy-alert-info.attention,.rst-content .wy-alert-info.caution,.rst-content .wy-alert-info.danger,.rst-content .wy-alert-info.error,.rst-content .wy-alert-info.hint,.rst-content .wy-alert-info.important,.rst-content .wy-alert-info.tip,.rst-content .wy-alert-info.warning,.wy-alert.wy-alert-info{background:#e7f2fa}.rst-content .note .admonition-title,.rst-content .note .wy-alert-title,.rst-content .seealso .admonition-title,.rst-content .seealso .wy-alert-title,.rst-content .wy-alert-info.admonition-todo .admonition-title,.rst-content .wy-alert-info.admonition-todo .wy-alert-title,.rst-content .wy-alert-info.admonition .admonition-title,.rst-content .wy-alert-info.admonition .wy-alert-title,.rst-content .wy-alert-info.attention .admonition-title,.rst-content .wy-alert-info.attention .wy-alert-title,.rst-content .wy-alert-info.caution .admonition-title,.rst-content .wy-alert-info.caution .wy-alert-title,.rst-content .wy-alert-info.danger .admonition-title,.rst-content .wy-alert-info.danger .wy-alert-title,.rst-content .wy-alert-info.error .admonition-title,.rst-content .wy-alert-info.error .wy-alert-title,.rst-content .wy-alert-info.hint .admonition-title,.rst-content .wy-alert-info.hint .wy-alert-title,.rst-content .wy-alert-info.important .admonition-title,.rst-content .wy-alert-info.important .wy-alert-title,.rst-content .wy-alert-info.tip .admonition-title,.rst-content .wy-alert-info.tip .wy-alert-title,.rst-content .wy-alert-info.warning .admonition-title,.rst-content .wy-alert-info.warning .wy-alert-title,.rst-content .wy-alert.wy-alert-info .admonition-title,.wy-alert.wy-alert-info .rst-content .admonition-title,.wy-alert.wy-alert-info .wy-alert-title{background:#6ab0de}.rst-content .hint,.rst-content .important,.rst-content .tip,.rst-content .wy-alert-success.admonition,.rst-content .wy-alert-success.admonition-todo,.rst-content .wy-alert-success.attention,.rst-content .wy-alert-success.caution,.rst-content .wy-alert-success.danger,.rst-content .wy-alert-success.error,.rst-content .wy-alert-success.note,.rst-content .wy-alert-success.seealso,.rst-content .wy-alert-success.warning,.wy-alert.wy-alert-success{background:#dbfaf4}.rst-content .hint .admonition-title,.rst-content .hint .wy-alert-title,.rst-content .important .admonition-title,.rst-content .important .wy-alert-title,.rst-content .tip .admonition-title,.rst-content .tip .wy-alert-title,.rst-content .wy-alert-success.admonition-todo .admonition-title,.rst-content .wy-alert-success.admonition-todo .wy-alert-title,.rst-content .wy-alert-success.admonition .admonition-title,.rst-content .wy-alert-success.admonition .wy-alert-title,.rst-content .wy-alert-success.attention .admonition-title,.rst-content .wy-alert-success.attention .wy-alert-title,.rst-content .wy-alert-success.caution .admonition-title,.rst-content .wy-alert-success.caution .wy-alert-title,.rst-content .wy-alert-success.danger .admonition-title,.rst-content .wy-alert-success.danger .wy-alert-title,.rst-content .wy-alert-success.error .admonition-title,.rst-content .wy-alert-success.error .wy-alert-title,.rst-content .wy-alert-success.note .admonition-title,.rst-content .wy-alert-success.note .wy-alert-title,.rst-content .wy-alert-success.seealso .admonition-title,.rst-content .wy-alert-success.seealso .wy-alert-title,.rst-content .wy-alert-success.warning .admonition-title,.rst-content .wy-alert-success.warning .wy-alert-title,.rst-content .wy-alert.wy-alert-success .admonition-title,.wy-alert.wy-alert-success .rst-content .admonition-title,.wy-alert.wy-alert-success .wy-alert-title{background:#1abc9c}.rst-content .wy-alert-neutral.admonition,.rst-content .wy-alert-neutral.admonition-todo,.rst-content .wy-alert-neutral.attention,.rst-content .wy-alert-neutral.caution,.rst-content .wy-alert-neutral.danger,.rst-content .wy-alert-neutral.error,.rst-content .wy-alert-neutral.hint,.rst-content .wy-alert-neutral.important,.rst-content .wy-alert-neutral.note,.rst-content .wy-alert-neutral.seealso,.rst-content .wy-alert-neutral.tip,.rst-content .wy-alert-neutral.warning,.wy-alert.wy-alert-neutral{background:#f3f6f6}.rst-content .wy-alert-neutral.admonition-todo .admonition-title,.rst-content .wy-alert-neutral.admonition-todo .wy-alert-title,.rst-content .wy-alert-neutral.admonition .admonition-title,.rst-content .wy-alert-neutral.admonition .wy-alert-title,.rst-content .wy-alert-neutral.attention .admonition-title,.rst-content .wy-alert-neutral.attention .wy-alert-title,.rst-content .wy-alert-neutral.caution .admonition-title,.rst-content .wy-alert-neutral.caution .wy-alert-title,.rst-content .wy-alert-neutral.danger .admonition-title,.rst-content .wy-alert-neutral.danger .wy-alert-title,.rst-content .wy-alert-neutral.error .admonition-title,.rst-content .wy-alert-neutral.error .wy-alert-title,.rst-content .wy-alert-neutral.hint .admonition-title,.rst-content .wy-alert-neutral.hint .wy-alert-title,.rst-content .wy-alert-neutral.important .admonition-title,.rst-content .wy-alert-neutral.important .wy-alert-title,.rst-content .wy-alert-neutral.note .admonition-title,.rst-content .wy-alert-neutral.note .wy-alert-title,.rst-content .wy-alert-neutral.seealso .admonition-title,.rst-content .wy-alert-neutral.seealso .wy-alert-title,.rst-content .wy-alert-neutral.tip .admonition-title,.rst-content .wy-alert-neutral.tip .wy-alert-title,.rst-content .wy-alert-neutral.warning .admonition-title,.rst-content .wy-alert-neutral.warning .wy-alert-title,.rst-content .wy-alert.wy-alert-neutral .admonition-title,.wy-alert.wy-alert-neutral .rst-content .admonition-title,.wy-alert.wy-alert-neutral .wy-alert-title{color:#404040;background:#e1e4e5}.rst-content .wy-alert-neutral.admonition-todo a,.rst-content .wy-alert-neutral.admonition a,.rst-content .wy-alert-neutral.attention a,.rst-content .wy-alert-neutral.caution a,.rst-content .wy-alert-neutral.danger a,.rst-content .wy-alert-neutral.error a,.rst-content .wy-alert-neutral.hint a,.rst-content .wy-alert-neutral.important a,.rst-content .wy-alert-neutral.note a,.rst-content .wy-alert-neutral.seealso a,.rst-content .wy-alert-neutral.tip a,.rst-content .wy-alert-neutral.warning a,.wy-alert.wy-alert-neutral a{color:#2980b9}.rst-content .admonition-todo p:last-child,.rst-content .admonition p:last-child,.rst-content .attention p:last-child,.rst-content .caution p:last-child,.rst-content .danger p:last-child,.rst-content .error p:last-child,.rst-content .hint p:last-child,.rst-content .important p:last-child,.rst-content .note p:last-child,.rst-content .seealso p:last-child,.rst-content .tip p:last-child,.rst-content .warning p:last-child,.wy-alert p:last-child{margin-bottom:0}.wy-tray-container{position:fixed;bottom:0;left:0;z-index:600}.wy-tray-container li{display:block;width:300px;background:transparent;color:#fff;text-align:center;box-shadow:0 5px 5px 0 rgba(0,0,0,.1);padding:0 24px;min-width:20%;opacity:0;height:0;line-height:56px;overflow:hidden;-webkit-transition:all .3s ease-in;-moz-transition:all .3s ease-in;transition:all .3s ease-in}.wy-tray-container li.wy-tray-item-success{background:#27ae60}.wy-tray-container li.wy-tray-item-info{background:#2980b9}.wy-tray-container li.wy-tray-item-warning{background:#e67e22}.wy-tray-container li.wy-tray-item-danger{background:#e74c3c}.wy-tray-container li.on{opacity:1;height:56px}@media screen and (max-width:768px){.wy-tray-container{bottom:auto;top:0;width:100%}.wy-tray-container li{width:100%}}button{font-size:100%;margin:0;vertical-align:baseline;*vertical-align:middle;cursor:pointer;line-height:normal;-webkit-appearance:button;*overflow:visible}button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}button[disabled]{cursor:default}.btn{display:inline-block;border-radius:2px;line-height:normal;white-space:nowrap;text-align:center;cursor:pointer;font-size:100%;padding:6px 12px 8px;color:#fff;border:1px solid rgba(0,0,0,.1);background-color:#27ae60;text-decoration:none;font-weight:400;font-family:Lato,proxima-nova,Helvetica Neue,Arial,sans-serif;box-shadow:inset 0 1px 2px -1px hsla(0,0%,100%,.5),inset 0 -2px 0 0 rgba(0,0,0,.1);outline-none:false;vertical-align:middle;*display:inline;zoom:1;-webkit-user-drag:none;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none;-webkit-transition:all .1s linear;-moz-transition:all .1s linear;transition:all .1s linear}.btn-hover{background:#2e8ece;color:#fff}.btn:hover{background:#2cc36b;color:#fff}.btn:focus{background:#2cc36b;outline:0}.btn:active{box-shadow:inset 0 -1px 0 0 rgba(0,0,0,.05),inset 0 2px 0 0 rgba(0,0,0,.1);padding:8px 12px 6px}.btn:visited{color:#fff}.btn-disabled,.btn-disabled:active,.btn-disabled:focus,.btn-disabled:hover,.btn:disabled{background-image:none;filter:progid:DXImageTransform.Microsoft.gradient(enabled = false);filter:alpha(opacity=40);opacity:.4;cursor:not-allowed;box-shadow:none}.btn::-moz-focus-inner{padding:0;border:0}.btn-small{font-size:80%}.btn-info{background-color:#2980b9!important}.btn-info:hover{background-color:#2e8ece!important}.btn-neutral{background-color:#f3f6f6!important;color:#404040!important}.btn-neutral:hover{background-color:#e5ebeb!important;color:#404040}.btn-neutral:visited{color:#404040!important}.btn-success{background-color:#27ae60!important}.btn-success:hover{background-color:#295!important}.btn-danger{background-color:#e74c3c!important}.btn-danger:hover{background-color:#ea6153!important}.btn-warning{background-color:#e67e22!important}.btn-warning:hover{background-color:#e98b39!important}.btn-invert{background-color:#222}.btn-invert:hover{background-color:#2f2f2f!important}.btn-link{background-color:transparent!important;color:#2980b9;box-shadow:none;border-color:transparent!important}.btn-link:active,.btn-link:hover{background-color:transparent!important;color:#409ad5!important;box-shadow:none}.btn-link:visited{color:#9b59b6}.wy-btn-group .btn,.wy-control .btn{vertical-align:middle}.wy-btn-group{margin-bottom:24px;*zoom:1}.wy-btn-group:after,.wy-btn-group:before{display:table;content:""}.wy-btn-group:after{clear:both}.wy-dropdown{position:relative;display:inline-block}.wy-dropdown-active .wy-dropdown-menu{display:block}.wy-dropdown-menu{position:absolute;left:0;display:none;float:left;top:100%;min-width:100%;background:#fcfcfc;z-index:100;border:1px solid #cfd7dd;box-shadow:0 2px 2px 0 rgba(0,0,0,.1);padding:12px}.wy-dropdown-menu>dd>a{display:block;clear:both;color:#404040;white-space:nowrap;font-size:90%;padding:0 12px;cursor:pointer}.wy-dropdown-menu>dd>a:hover{background:#2980b9;color:#fff}.wy-dropdown-menu>dd.divider{border-top:1px solid #cfd7dd;margin:6px 0}.wy-dropdown-menu>dd.search{padding-bottom:12px}.wy-dropdown-menu>dd.search input[type=search]{width:100%}.wy-dropdown-menu>dd.call-to-action{background:#e3e3e3;text-transform:uppercase;font-weight:500;font-size:80%}.wy-dropdown-menu>dd.call-to-action:hover{background:#e3e3e3}.wy-dropdown-menu>dd.call-to-action .btn{color:#fff}.wy-dropdown.wy-dropdown-up .wy-dropdown-menu{bottom:100%;top:auto;left:auto;right:0}.wy-dropdown.wy-dropdown-bubble .wy-dropdown-menu{background:#fcfcfc;margin-top:2px}.wy-dropdown.wy-dropdown-bubble .wy-dropdown-menu a{padding:6px 12px}.wy-dropdown.wy-dropdown-bubble .wy-dropdown-menu a:hover{background:#2980b9;color:#fff}.wy-dropdown.wy-dropdown-left .wy-dropdown-menu{right:0;left:auto;text-align:right}.wy-dropdown-arrow:before{content:" ";border-bottom:5px solid #f5f5f5;border-left:5px solid transparent;border-right:5px solid transparent;position:absolute;display:block;top:-4px;left:50%;margin-left:-3px}.wy-dropdown-arrow.wy-dropdown-arrow-left:before{left:11px}.wy-form-stacked select{display:block}.wy-form-aligned .wy-help-inline,.wy-form-aligned input,.wy-form-aligned label,.wy-form-aligned select,.wy-form-aligned textarea{display:inline-block;*display:inline;*zoom:1;vertical-align:middle}.wy-form-aligned .wy-control-group>label{display:inline-block;vertical-align:middle;width:10em;margin:6px 12px 0 0;float:left}.wy-form-aligned .wy-control{float:left}.wy-form-aligned .wy-control label{display:block}.wy-form-aligned .wy-control select{margin-top:6px}fieldset{margin:0}fieldset,legend{border:0;padding:0}legend{width:100%;white-space:normal;margin-bottom:24px;font-size:150%;*margin-left:-7px}label,legend{display:block}label{margin:0 0 .3125em;color:#333;font-size:90%}input,select,textarea{font-size:100%;margin:0;vertical-align:baseline;*vertical-align:middle}.wy-control-group{margin-bottom:24px;max-width:1200px;margin-left:auto;margin-right:auto;*zoom:1}.wy-control-group:after,.wy-control-group:before{display:table;content:""}.wy-control-group:after{clear:both}.wy-control-group.wy-control-group-required>label:after{content:" *";color:#e74c3c}.wy-control-group .wy-form-full,.wy-control-group .wy-form-halves,.wy-control-group .wy-form-thirds{padding-bottom:12px}.wy-control-group .wy-form-full input[type=color],.wy-control-group .wy-form-full input[type=date],.wy-control-group .wy-form-full input[type=datetime-local],.wy-control-group .wy-form-full input[type=datetime],.wy-control-group .wy-form-full input[type=email],.wy-control-group .wy-form-full input[type=month],.wy-control-group .wy-form-full input[type=number],.wy-control-group .wy-form-full input[type=password],.wy-control-group .wy-form-full input[type=search],.wy-control-group .wy-form-full input[type=tel],.wy-control-group .wy-form-full input[type=text],.wy-control-group .wy-form-full input[type=time],.wy-control-group .wy-form-full input[type=url],.wy-control-group .wy-form-full input[type=week],.wy-control-group .wy-form-full select,.wy-control-group .wy-form-halves input[type=color],.wy-control-group .wy-form-halves input[type=date],.wy-control-group .wy-form-halves input[type=datetime-local],.wy-control-group .wy-form-halves input[type=datetime],.wy-control-group .wy-form-halves input[type=email],.wy-control-group .wy-form-halves input[type=month],.wy-control-group .wy-form-halves input[type=number],.wy-control-group .wy-form-halves input[type=password],.wy-control-group .wy-form-halves input[type=search],.wy-control-group .wy-form-halves input[type=tel],.wy-control-group .wy-form-halves input[type=text],.wy-control-group .wy-form-halves input[type=time],.wy-control-group .wy-form-halves input[type=url],.wy-control-group .wy-form-halves input[type=week],.wy-control-group .wy-form-halves select,.wy-control-group .wy-form-thirds input[type=color],.wy-control-group .wy-form-thirds input[type=date],.wy-control-group .wy-form-thirds input[type=datetime-local],.wy-control-group .wy-form-thirds input[type=datetime],.wy-control-group .wy-form-thirds input[type=email],.wy-control-group .wy-form-thirds input[type=month],.wy-control-group .wy-form-thirds input[type=number],.wy-control-group .wy-form-thirds input[type=password],.wy-control-group .wy-form-thirds input[type=search],.wy-control-group .wy-form-thirds input[type=tel],.wy-control-group .wy-form-thirds input[type=text],.wy-control-group .wy-form-thirds input[type=time],.wy-control-group .wy-form-thirds input[type=url],.wy-control-group .wy-form-thirds input[type=week],.wy-control-group .wy-form-thirds select{width:100%}.wy-control-group .wy-form-full{float:left;display:block;width:100%;margin-right:0}.wy-control-group .wy-form-full:last-child{margin-right:0}.wy-control-group .wy-form-halves{float:left;display:block;margin-right:2.35765%;width:48.82117%}.wy-control-group .wy-form-halves:last-child,.wy-control-group .wy-form-halves:nth-of-type(2n){margin-right:0}.wy-control-group .wy-form-halves:nth-of-type(odd){clear:left}.wy-control-group .wy-form-thirds{float:left;display:block;margin-right:2.35765%;width:31.76157%}.wy-control-group .wy-form-thirds:last-child,.wy-control-group .wy-form-thirds:nth-of-type(3n){margin-right:0}.wy-control-group .wy-form-thirds:nth-of-type(3n+1){clear:left}.wy-control-group.wy-control-group-no-input .wy-control,.wy-control-no-input{margin:6px 0 0;font-size:90%}.wy-control-no-input{display:inline-block}.wy-control-group.fluid-input input[type=color],.wy-control-group.fluid-input input[type=date],.wy-control-group.fluid-input input[type=datetime-local],.wy-control-group.fluid-input input[type=datetime],.wy-control-group.fluid-input input[type=email],.wy-control-group.fluid-input input[type=month],.wy-control-group.fluid-input input[type=number],.wy-control-group.fluid-input input[type=password],.wy-control-group.fluid-input input[type=search],.wy-control-group.fluid-input input[type=tel],.wy-control-group.fluid-input input[type=text],.wy-control-group.fluid-input input[type=time],.wy-control-group.fluid-input input[type=url],.wy-control-group.fluid-input input[type=week]{width:100%}.wy-form-message-inline{padding-left:.3em;color:#666;font-size:90%}.wy-form-message{display:block;color:#999;font-size:70%;margin-top:.3125em;font-style:italic}.wy-form-message p{font-size:inherit;font-style:italic;margin-bottom:6px}.wy-form-message p:last-child{margin-bottom:0}input{line-height:normal}input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer;font-family:Lato,proxima-nova,Helvetica Neue,Arial,sans-serif;*overflow:visible}input[type=color],input[type=date],input[type=datetime-local],input[type=datetime],input[type=email],input[type=month],input[type=number],input[type=password],input[type=search],input[type=tel],input[type=text],input[type=time],input[type=url],input[type=week]{-webkit-appearance:none;padding:6px;display:inline-block;border:1px solid #ccc;font-size:80%;font-family:Lato,proxima-nova,Helvetica Neue,Arial,sans-serif;box-shadow:inset 0 1px 3px #ddd;border-radius:0;-webkit-transition:border .3s linear;-moz-transition:border .3s linear;transition:border .3s linear}input[type=datetime-local]{padding:.34375em .625em}input[disabled]{cursor:default}input[type=checkbox],input[type=radio]{padding:0;margin-right:.3125em;*height:13px;*width:13px}input[type=checkbox],input[type=radio],input[type=search]{-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}input[type=search]::-webkit-search-cancel-button,input[type=search]::-webkit-search-decoration{-webkit-appearance:none}input[type=color]:focus,input[type=date]:focus,input[type=datetime-local]:focus,input[type=datetime]:focus,input[type=email]:focus,input[type=month]:focus,input[type=number]:focus,input[type=password]:focus,input[type=search]:focus,input[type=tel]:focus,input[type=text]:focus,input[type=time]:focus,input[type=url]:focus,input[type=week]:focus{outline:0;outline:thin dotted\9;border-color:#333}input.no-focus:focus{border-color:#ccc!important}input[type=checkbox]:focus,input[type=file]:focus,input[type=radio]:focus{outline:thin dotted #333;outline:1px auto #129fea}input[type=color][disabled],input[type=date][disabled],input[type=datetime-local][disabled],input[type=datetime][disabled],input[type=email][disabled],input[type=month][disabled],input[type=number][disabled],input[type=password][disabled],input[type=search][disabled],input[type=tel][disabled],input[type=text][disabled],input[type=time][disabled],input[type=url][disabled],input[type=week][disabled]{cursor:not-allowed;background-color:#fafafa}input:focus:invalid,select:focus:invalid,textarea:focus:invalid{color:#e74c3c;border:1px solid #e74c3c}input:focus:invalid:focus,select:focus:invalid:focus,textarea:focus:invalid:focus{border-color:#e74c3c}input[type=checkbox]:focus:invalid:focus,input[type=file]:focus:invalid:focus,input[type=radio]:focus:invalid:focus{outline-color:#e74c3c}input.wy-input-large{padding:12px;font-size:100%}textarea{overflow:auto;vertical-align:top;width:100%;font-family:Lato,proxima-nova,Helvetica Neue,Arial,sans-serif}select,textarea{padding:.5em .625em;display:inline-block;border:1px solid #ccc;font-size:80%;box-shadow:inset 0 1px 3px #ddd;-webkit-transition:border .3s linear;-moz-transition:border .3s linear;transition:border .3s linear}select{border:1px solid #ccc;background-color:#fff}select[multiple]{height:auto}select:focus,textarea:focus{outline:0}input[readonly],select[disabled],select[readonly],textarea[disabled],textarea[readonly]{cursor:not-allowed;background-color:#fafafa}input[type=checkbox][disabled],input[type=radio][disabled]{cursor:not-allowed}.wy-checkbox,.wy-radio{margin:6px 0;color:#404040;display:block}.wy-checkbox input,.wy-radio input{vertical-align:baseline}.wy-form-message-inline{display:inline-block;*display:inline;*zoom:1;vertical-align:middle}.wy-input-prefix,.wy-input-suffix{white-space:nowrap;padding:6px}.wy-input-prefix .wy-input-context,.wy-input-suffix .wy-input-context{line-height:27px;padding:0 8px;display:inline-block;font-size:80%;background-color:#f3f6f6;border:1px solid #ccc;color:#999}.wy-input-suffix .wy-input-context{border-left:0}.wy-input-prefix .wy-input-context{border-right:0}.wy-switch{position:relative;display:block;height:24px;margin-top:12px;cursor:pointer}.wy-switch:before{left:0;top:0;width:36px;height:12px;background:#ccc}.wy-switch:after,.wy-switch:before{position:absolute;content:"";display:block;border-radius:4px;-webkit-transition:all .2s ease-in-out;-moz-transition:all .2s ease-in-out;transition:all .2s ease-in-out}.wy-switch:after{width:18px;height:18px;background:#999;left:-3px;top:-3px}.wy-switch span{position:absolute;left:48px;display:block;font-size:12px;color:#ccc;line-height:1}.wy-switch.active:before{background:#1e8449}.wy-switch.active:after{left:24px;background:#27ae60}.wy-switch.disabled{cursor:not-allowed;opacity:.8}.wy-control-group.wy-control-group-error .wy-form-message,.wy-control-group.wy-control-group-error>label{color:#e74c3c}.wy-control-group.wy-control-group-error input[type=color],.wy-control-group.wy-control-group-error input[type=date],.wy-control-group.wy-control-group-error input[type=datetime-local],.wy-control-group.wy-control-group-error input[type=datetime],.wy-control-group.wy-control-group-error input[type=email],.wy-control-group.wy-control-group-error input[type=month],.wy-control-group.wy-control-group-error input[type=number],.wy-control-group.wy-control-group-error input[type=password],.wy-control-group.wy-control-group-error input[type=search],.wy-control-group.wy-control-group-error input[type=tel],.wy-control-group.wy-control-group-error input[type=text],.wy-control-group.wy-control-group-error input[type=time],.wy-control-group.wy-control-group-error input[type=url],.wy-control-group.wy-control-group-error input[type=week],.wy-control-group.wy-control-group-error textarea{border:1px solid #e74c3c}.wy-inline-validate{white-space:nowrap}.wy-inline-validate .wy-input-context{padding:.5em .625em;display:inline-block;font-size:80%}.wy-inline-validate.wy-inline-validate-success .wy-input-context{color:#27ae60}.wy-inline-validate.wy-inline-validate-danger .wy-input-context{color:#e74c3c}.wy-inline-validate.wy-inline-validate-warning .wy-input-context{color:#e67e22}.wy-inline-validate.wy-inline-validate-info .wy-input-context{color:#2980b9}.rotate-90{-webkit-transform:rotate(90deg);-moz-transform:rotate(90deg);-ms-transform:rotate(90deg);-o-transform:rotate(90deg);transform:rotate(90deg)}.rotate-180{-webkit-transform:rotate(180deg);-moz-transform:rotate(180deg);-ms-transform:rotate(180deg);-o-transform:rotate(180deg);transform:rotate(180deg)}.rotate-270{-webkit-transform:rotate(270deg);-moz-transform:rotate(270deg);-ms-transform:rotate(270deg);-o-transform:rotate(270deg);transform:rotate(270deg)}.mirror{-webkit-transform:scaleX(-1);-moz-transform:scaleX(-1);-ms-transform:scaleX(-1);-o-transform:scaleX(-1);transform:scaleX(-1)}.mirror.rotate-90{-webkit-transform:scaleX(-1) rotate(90deg);-moz-transform:scaleX(-1) rotate(90deg);-ms-transform:scaleX(-1) rotate(90deg);-o-transform:scaleX(-1) rotate(90deg);transform:scaleX(-1) rotate(90deg)}.mirror.rotate-180{-webkit-transform:scaleX(-1) rotate(180deg);-moz-transform:scaleX(-1) rotate(180deg);-ms-transform:scaleX(-1) rotate(180deg);-o-transform:scaleX(-1) rotate(180deg);transform:scaleX(-1) rotate(180deg)}.mirror.rotate-270{-webkit-transform:scaleX(-1) rotate(270deg);-moz-transform:scaleX(-1) rotate(270deg);-ms-transform:scaleX(-1) rotate(270deg);-o-transform:scaleX(-1) rotate(270deg);transform:scaleX(-1) rotate(270deg)}@media only screen and (max-width:480px){.wy-form button[type=submit]{margin:.7em 0 0}.wy-form input[type=color],.wy-form input[type=date],.wy-form input[type=datetime-local],.wy-form input[type=datetime],.wy-form input[type=email],.wy-form input[type=month],.wy-form input[type=number],.wy-form input[type=password],.wy-form input[type=search],.wy-form input[type=tel],.wy-form input[type=text],.wy-form input[type=time],.wy-form input[type=url],.wy-form input[type=week],.wy-form label{margin-bottom:.3em;display:block}.wy-form input[type=color],.wy-form input[type=date],.wy-form input[type=datetime-local],.wy-form input[type=datetime],.wy-form input[type=email],.wy-form input[type=month],.wy-form input[type=number],.wy-form input[type=password],.wy-form input[type=search],.wy-form input[type=tel],.wy-form input[type=time],.wy-form input[type=url],.wy-form input[type=week]{margin-bottom:0}.wy-form-aligned .wy-control-group label{margin-bottom:.3em;text-align:left;display:block;width:100%}.wy-form-aligned .wy-control{margin:1.5em 0 0}.wy-form-message,.wy-form-message-inline,.wy-form .wy-help-inline{display:block;font-size:80%;padding:6px 0}}@media screen and (max-width:768px){.tablet-hide{display:none}}@media screen and (max-width:480px){.mobile-hide{display:none}}.float-left{float:left}.float-right{float:right}.full-width{width:100%}.rst-content table.docutils,.rst-content table.field-list,.wy-table{border-collapse:collapse;border-spacing:0;empty-cells:show;margin-bottom:24px}.rst-content table.docutils caption,.rst-content table.field-list caption,.wy-table caption{color:#000;font:italic 85%/1 arial,sans-serif;padding:1em 0;text-align:center}.rst-content table.docutils td,.rst-content table.docutils th,.rst-content table.field-list td,.rst-content table.field-list th,.wy-table td,.wy-table th{font-size:90%;margin:0;overflow:visible;padding:8px 16px}.rst-content table.docutils td:first-child,.rst-content table.docutils th:first-child,.rst-content table.field-list td:first-child,.rst-content table.field-list th:first-child,.wy-table td:first-child,.wy-table th:first-child{border-left-width:0}.rst-content table.docutils thead,.rst-content table.field-list thead,.wy-table thead{color:#000;text-align:left;vertical-align:bottom;white-space:nowrap}.rst-content table.docutils thead th,.rst-content table.field-list thead th,.wy-table thead th{font-weight:700;border-bottom:2px solid #e1e4e5}.rst-content table.docutils td,.rst-content table.field-list td,.wy-table td{background-color:transparent;vertical-align:middle}.rst-content table.docutils td p,.rst-content table.field-list td p,.wy-table td p{line-height:18px}.rst-content table.docutils td p:last-child,.rst-content table.field-list td p:last-child,.wy-table td p:last-child{margin-bottom:0}.rst-content table.docutils .wy-table-cell-min,.rst-content table.field-list .wy-table-cell-min,.wy-table .wy-table-cell-min{width:1%;padding-right:0}.rst-content table.docutils .wy-table-cell-min input[type=checkbox],.rst-content table.field-list .wy-table-cell-min input[type=checkbox],.wy-table .wy-table-cell-min input[type=checkbox]{margin:0}.wy-table-secondary{color:grey;font-size:90%}.wy-table-tertiary{color:grey;font-size:80%}.rst-content table.docutils:not(.field-list) tr:nth-child(2n-1) td,.wy-table-backed,.wy-table-odd td,.wy-table-striped tr:nth-child(2n-1) td{background-color:#f3f6f6}.rst-content table.docutils,.wy-table-bordered-all{border:1px solid #e1e4e5}.rst-content table.docutils td,.wy-table-bordered-all td{border-bottom:1px solid #e1e4e5;border-left:1px solid #e1e4e5}.rst-content table.docutils tbody>tr:last-child td,.wy-table-bordered-all tbody>tr:last-child td{border-bottom-width:0}.wy-table-bordered{border:1px solid #e1e4e5}.wy-table-bordered-rows td{border-bottom:1px solid #e1e4e5}.wy-table-bordered-rows tbody>tr:last-child td{border-bottom-width:0}.wy-table-horizontal td,.wy-table-horizontal th{border-width:0 0 1px;border-bottom:1px solid #e1e4e5}.wy-table-horizontal tbody>tr:last-child td{border-bottom-width:0}.wy-table-responsive{margin-bottom:24px;max-width:100%;overflow:auto}.wy-table-responsive table{margin-bottom:0!important}.wy-table-responsive table td,.wy-table-responsive table th{white-space:nowrap}a{color:#2980b9;text-decoration:none;cursor:pointer}a:hover{color:#3091d1}a:visited{color:#9b59b6}html{height:100%}body,html{overflow-x:hidden}body{font-family:Lato,proxima-nova,Helvetica Neue,Arial,sans-serif;font-weight:400;color:#404040;min-height:100%;background:#edf0f2}.wy-text-left{text-align:left}.wy-text-center{text-align:center}.wy-text-right{text-align:right}.wy-text-large{font-size:120%}.wy-text-normal{font-size:100%}.wy-text-small,small{font-size:80%}.wy-text-strike{text-decoration:line-through}.wy-text-warning{color:#e67e22!important}a.wy-text-warning:hover{color:#eb9950!important}.wy-text-info{color:#2980b9!important}a.wy-text-info:hover{color:#409ad5!important}.wy-text-success{color:#27ae60!important}a.wy-text-success:hover{color:#36d278!important}.wy-text-danger{color:#e74c3c!important}a.wy-text-danger:hover{color:#ed7669!important}.wy-text-neutral{color:#404040!important}a.wy-text-neutral:hover{color:#595959!important}.rst-content .toctree-wrapper>p.caption,h1,h2,h3,h4,h5,h6,legend{margin-top:0;font-weight:700;font-family:Roboto Slab,ff-tisa-web-pro,Georgia,Arial,sans-serif}p{line-height:24px;font-size:16px;margin:0 0 24px}h1{font-size:175%}.rst-content .toctree-wrapper>p.caption,h2{font-size:150%}h3{font-size:125%}h4{font-size:115%}h5{font-size:110%}h6{font-size:100%}hr{display:block;height:1px;border:0;border-top:1px solid #e1e4e5;margin:24px 0;padding:0}.rst-content code,.rst-content tt,code{white-space:nowrap;max-width:100%;background:#fff;border:1px solid #e1e4e5;font-size:75%;padding:0 5px;font-family:SFMono-Regular,Menlo,Monaco,Consolas,Liberation Mono,Courier New,Courier,monospace;color:#e74c3c;overflow-x:auto}.rst-content tt.code-large,code.code-large{font-size:90%}.rst-content .section ul,.rst-content .toctree-wrapper ul,.rst-content section ul,.wy-plain-list-disc,article ul{list-style:disc;line-height:24px;margin-bottom:24px}.rst-content .section ul li,.rst-content .toctree-wrapper ul li,.rst-content section ul li,.wy-plain-list-disc li,article ul li{list-style:disc;margin-left:24px}.rst-content .section ul li p:last-child,.rst-content .section ul li ul,.rst-content .toctree-wrapper ul li p:last-child,.rst-content .toctree-wrapper ul li ul,.rst-content section ul li p:last-child,.rst-content section ul li ul,.wy-plain-list-disc li p:last-child,.wy-plain-list-disc li ul,article ul li p:last-child,article ul li ul{margin-bottom:0}.rst-content .section ul li li,.rst-content .toctree-wrapper ul li li,.rst-content section ul li li,.wy-plain-list-disc li li,article ul li li{list-style:circle}.rst-content .section ul li li li,.rst-content .toctree-wrapper ul li li li,.rst-content section ul li li li,.wy-plain-list-disc li li li,article ul li li li{list-style:square}.rst-content .section ul li ol li,.rst-content .toctree-wrapper ul li ol li,.rst-content section ul li ol li,.wy-plain-list-disc li ol li,article ul li ol li{list-style:decimal}.rst-content .section ol,.rst-content .section ol.arabic,.rst-content .toctree-wrapper ol,.rst-content .toctree-wrapper ol.arabic,.rst-content section ol,.rst-content section ol.arabic,.wy-plain-list-decimal,article ol{list-style:decimal;line-height:24px;margin-bottom:24px}.rst-content .section ol.arabic li,.rst-content .section ol li,.rst-content .toctree-wrapper ol.arabic li,.rst-content .toctree-wrapper ol li,.rst-content section ol.arabic li,.rst-content section ol li,.wy-plain-list-decimal li,article ol li{list-style:decimal;margin-left:24px}.rst-content .section ol.arabic li ul,.rst-content .section ol li p:last-child,.rst-content .section ol li ul,.rst-content .toctree-wrapper ol.arabic li ul,.rst-content .toctree-wrapper ol li p:last-child,.rst-content .toctree-wrapper ol li ul,.rst-content section ol.arabic li ul,.rst-content section ol li p:last-child,.rst-content section ol li ul,.wy-plain-list-decimal li p:last-child,.wy-plain-list-decimal li ul,article ol li p:last-child,article ol li ul{margin-bottom:0}.rst-content .section ol.arabic li ul li,.rst-content .section ol li ul li,.rst-content .toctree-wrapper ol.arabic li ul li,.rst-content .toctree-wrapper ol li ul li,.rst-content section ol.arabic li ul li,.rst-content section ol li ul li,.wy-plain-list-decimal li ul li,article ol li ul li{list-style:disc}.wy-breadcrumbs{*zoom:1}.wy-breadcrumbs:after,.wy-breadcrumbs:before{display:table;content:""}.wy-breadcrumbs:after{clear:both}.wy-breadcrumbs>li{display:inline-block;padding-top:5px}.wy-breadcrumbs>li.wy-breadcrumbs-aside{float:right}.rst-content .wy-breadcrumbs>li code,.rst-content .wy-breadcrumbs>li tt,.wy-breadcrumbs>li .rst-content tt,.wy-breadcrumbs>li code{all:inherit;color:inherit}.breadcrumb-item:before{content:"/";color:#bbb;font-size:13px;padding:0 6px 0 3px}.wy-breadcrumbs-extra{margin-bottom:0;color:#b3b3b3;font-size:80%;display:inline-block}@media screen and (max-width:480px){.wy-breadcrumbs-extra,.wy-breadcrumbs li.wy-breadcrumbs-aside{display:none}}@media print{.wy-breadcrumbs li.wy-breadcrumbs-aside{display:none}}html{font-size:16px}.wy-affix{position:fixed;top:1.618em}.wy-menu a:hover{text-decoration:none}.wy-menu-horiz{*zoom:1}.wy-menu-horiz:after,.wy-menu-horiz:before{display:table;content:""}.wy-menu-horiz:after{clear:both}.wy-menu-horiz li,.wy-menu-horiz ul{display:inline-block}.wy-menu-horiz li:hover{background:hsla(0,0%,100%,.1)}.wy-menu-horiz li.divide-left{border-left:1px solid #404040}.wy-menu-horiz li.divide-right{border-right:1px solid #404040}.wy-menu-horiz a{height:32px;display:inline-block;line-height:32px;padding:0 16px}.wy-menu-vertical{width:300px}.wy-menu-vertical header,.wy-menu-vertical p.caption{color:#55a5d9;height:32px;line-height:32px;padding:0 1.618em;margin:12px 0 0;display:block;font-weight:700;text-transform:uppercase;font-size:85%;white-space:nowrap}.wy-menu-vertical ul{margin-bottom:0}.wy-menu-vertical li.divide-top{border-top:1px solid #404040}.wy-menu-vertical li.divide-bottom{border-bottom:1px solid #404040}.wy-menu-vertical li.current{background:#e3e3e3}.wy-menu-vertical li.current a{color:grey;border-right:1px solid #c9c9c9;padding:.4045em 2.427em}.wy-menu-vertical li.current a:hover{background:#d6d6d6}.rst-content .wy-menu-vertical li tt,.wy-menu-vertical li .rst-content tt,.wy-menu-vertical li code{border:none;background:inherit;color:inherit;padding-left:0;padding-right:0}.wy-menu-vertical li button.toctree-expand{display:block;float:left;margin-left:-1.2em;line-height:18px;color:#4d4d4d;border:none;background:none;padding:0}.wy-menu-vertical li.current>a,.wy-menu-vertical li.on a{color:#404040;font-weight:700;position:relative;background:#fcfcfc;border:none;padding:.4045em 1.618em}.wy-menu-vertical li.current>a:hover,.wy-menu-vertical li.on a:hover{background:#fcfcfc}.wy-menu-vertical li.current>a:hover button.toctree-expand,.wy-menu-vertical li.on a:hover button.toctree-expand{color:grey}.wy-menu-vertical li.current>a button.toctree-expand,.wy-menu-vertical li.on a button.toctree-expand{display:block;line-height:18px;color:#333}.wy-menu-vertical li.toctree-l1.current>a{border-bottom:1px solid #c9c9c9;border-top:1px solid #c9c9c9}.wy-menu-vertical .toctree-l1.current .toctree-l2>ul,.wy-menu-vertical .toctree-l2.current .toctree-l3>ul,.wy-menu-vertical .toctree-l3.current .toctree-l4>ul,.wy-menu-vertical .toctree-l4.current .toctree-l5>ul,.wy-menu-vertical .toctree-l5.current .toctree-l6>ul,.wy-menu-vertical .toctree-l6.current .toctree-l7>ul,.wy-menu-vertical .toctree-l7.current .toctree-l8>ul,.wy-menu-vertical .toctree-l8.current .toctree-l9>ul,.wy-menu-vertical .toctree-l9.current .toctree-l10>ul,.wy-menu-vertical .toctree-l10.current .toctree-l11>ul{display:none}.wy-menu-vertical .toctree-l1.current .current.toctree-l2>ul,.wy-menu-vertical .toctree-l2.current .current.toctree-l3>ul,.wy-menu-vertical .toctree-l3.current .current.toctree-l4>ul,.wy-menu-vertical .toctree-l4.current .current.toctree-l5>ul,.wy-menu-vertical .toctree-l5.current .current.toctree-l6>ul,.wy-menu-vertical .toctree-l6.current .current.toctree-l7>ul,.wy-menu-vertical .toctree-l7.current .current.toctree-l8>ul,.wy-menu-vertical .toctree-l8.current .current.toctree-l9>ul,.wy-menu-vertical .toctree-l9.current .current.toctree-l10>ul,.wy-menu-vertical .toctree-l10.current .current.toctree-l11>ul{display:block}.wy-menu-vertical li.toctree-l3,.wy-menu-vertical li.toctree-l4{font-size:.9em}.wy-menu-vertical li.toctree-l2 a,.wy-menu-vertical li.toctree-l3 a,.wy-menu-vertical li.toctree-l4 a,.wy-menu-vertical li.toctree-l5 a,.wy-menu-vertical li.toctree-l6 a,.wy-menu-vertical li.toctree-l7 a,.wy-menu-vertical li.toctree-l8 a,.wy-menu-vertical li.toctree-l9 a,.wy-menu-vertical li.toctree-l10 a{color:#404040}.wy-menu-vertical li.toctree-l2 a:hover button.toctree-expand,.wy-menu-vertical li.toctree-l3 a:hover button.toctree-expand,.wy-menu-vertical li.toctree-l4 a:hover button.toctree-expand,.wy-menu-vertical li.toctree-l5 a:hover button.toctree-expand,.wy-menu-vertical li.toctree-l6 a:hover button.toctree-expand,.wy-menu-vertical li.toctree-l7 a:hover button.toctree-expand,.wy-menu-vertical li.toctree-l8 a:hover button.toctree-expand,.wy-menu-vertical li.toctree-l9 a:hover button.toctree-expand,.wy-menu-vertical li.toctree-l10 a:hover button.toctree-expand{color:grey}.wy-menu-vertical li.toctree-l2.current li.toctree-l3>a,.wy-menu-vertical li.toctree-l3.current li.toctree-l4>a,.wy-menu-vertical li.toctree-l4.current li.toctree-l5>a,.wy-menu-vertical li.toctree-l5.current li.toctree-l6>a,.wy-menu-vertical li.toctree-l6.current li.toctree-l7>a,.wy-menu-vertical li.toctree-l7.current li.toctree-l8>a,.wy-menu-vertical li.toctree-l8.current li.toctree-l9>a,.wy-menu-vertical li.toctree-l9.current li.toctree-l10>a,.wy-menu-vertical li.toctree-l10.current li.toctree-l11>a{display:block}.wy-menu-vertical li.toctree-l2.current>a{padding:.4045em 2.427em}.wy-menu-vertical li.toctree-l2.current li.toctree-l3>a{padding:.4045em 1.618em .4045em 4.045em}.wy-menu-vertical li.toctree-l3.current>a{padding:.4045em 4.045em}.wy-menu-vertical li.toctree-l3.current li.toctree-l4>a{padding:.4045em 1.618em .4045em 5.663em}.wy-menu-vertical li.toctree-l4.current>a{padding:.4045em 5.663em}.wy-menu-vertical li.toctree-l4.current li.toctree-l5>a{padding:.4045em 1.618em .4045em 7.281em}.wy-menu-vertical li.toctree-l5.current>a{padding:.4045em 7.281em}.wy-menu-vertical li.toctree-l5.current li.toctree-l6>a{padding:.4045em 1.618em .4045em 8.899em}.wy-menu-vertical li.toctree-l6.current>a{padding:.4045em 8.899em}.wy-menu-vertical li.toctree-l6.current li.toctree-l7>a{padding:.4045em 1.618em .4045em 10.517em}.wy-menu-vertical li.toctree-l7.current>a{padding:.4045em 10.517em}.wy-menu-vertical li.toctree-l7.current li.toctree-l8>a{padding:.4045em 1.618em .4045em 12.135em}.wy-menu-vertical li.toctree-l8.current>a{padding:.4045em 12.135em}.wy-menu-vertical li.toctree-l8.current li.toctree-l9>a{padding:.4045em 1.618em .4045em 13.753em}.wy-menu-vertical li.toctree-l9.current>a{padding:.4045em 13.753em}.wy-menu-vertical li.toctree-l9.current li.toctree-l10>a{padding:.4045em 1.618em .4045em 15.371em}.wy-menu-vertical li.toctree-l10.current>a{padding:.4045em 15.371em}.wy-menu-vertical li.toctree-l10.current li.toctree-l11>a{padding:.4045em 1.618em .4045em 16.989em}.wy-menu-vertical li.toctree-l2.current>a,.wy-menu-vertical li.toctree-l2.current li.toctree-l3>a{background:#c9c9c9}.wy-menu-vertical li.toctree-l2 button.toctree-expand{color:#a3a3a3}.wy-menu-vertical li.toctree-l3.current>a,.wy-menu-vertical li.toctree-l3.current li.toctree-l4>a{background:#bdbdbd}.wy-menu-vertical li.toctree-l3 button.toctree-expand{color:#969696}.wy-menu-vertical li.current ul{display:block}.wy-menu-vertical li ul{margin-bottom:0;display:none}.wy-menu-vertical li ul li a{margin-bottom:0;color:#d9d9d9;font-weight:400}.wy-menu-vertical a{line-height:18px;padding:.4045em 1.618em;display:block;position:relative;font-size:90%;color:#d9d9d9}.wy-menu-vertical a:hover{background-color:#4e4a4a;cursor:pointer}.wy-menu-vertical a:hover button.toctree-expand{color:#d9d9d9}.wy-menu-vertical a:active{background-color:#2980b9;cursor:pointer;color:#fff}.wy-menu-vertical a:active button.toctree-expand{color:#fff}.wy-side-nav-search{display:block;width:300px;padding:.809em;margin-bottom:.809em;z-index:200;background-color:#2980b9;text-align:center;color:#fcfcfc}.wy-side-nav-search input[type=text]{width:100%;border-radius:50px;padding:6px 12px;border-color:#2472a4}.wy-side-nav-search img{display:block;margin:auto auto .809em;height:45px;width:45px;background-color:#2980b9;padding:5px;border-radius:100%}.wy-side-nav-search .wy-dropdown>a,.wy-side-nav-search>a{color:#fcfcfc;font-size:100%;font-weight:700;display:inline-block;padding:4px 6px;margin-bottom:.809em;max-width:100%}.wy-side-nav-search .wy-dropdown>a:hover,.wy-side-nav-search>a:hover{background:hsla(0,0%,100%,.1)}.wy-side-nav-search .wy-dropdown>a img.logo,.wy-side-nav-search>a img.logo{display:block;margin:0 auto;height:auto;width:auto;border-radius:0;max-width:100%;background:transparent}.wy-side-nav-search .wy-dropdown>a.icon img.logo,.wy-side-nav-search>a.icon img.logo{margin-top:.85em}.wy-side-nav-search>div.version{margin-top:-.4045em;margin-bottom:.809em;font-weight:400;color:hsla(0,0%,100%,.3)}.wy-nav .wy-menu-vertical header{color:#2980b9}.wy-nav .wy-menu-vertical a{color:#b3b3b3}.wy-nav .wy-menu-vertical a:hover{background-color:#2980b9;color:#fff}[data-menu-wrap]{-webkit-transition:all .2s ease-in;-moz-transition:all .2s ease-in;transition:all .2s ease-in;position:absolute;opacity:1;width:100%;opacity:0}[data-menu-wrap].move-center{left:0;right:auto;opacity:1}[data-menu-wrap].move-left{right:auto;left:-100%;opacity:0}[data-menu-wrap].move-right{right:-100%;left:auto;opacity:0}.wy-body-for-nav{background:#fcfcfc}.wy-grid-for-nav{position:absolute;width:100%;height:100%}.wy-nav-side{position:fixed;top:0;bottom:0;left:0;padding-bottom:2em;width:300px;overflow-x:hidden;overflow-y:hidden;min-height:100%;color:#9b9b9b;background:#343131;z-index:200}.wy-side-scroll{width:320px;position:relative;overflow-x:hidden;overflow-y:scroll;height:100%}.wy-nav-top{display:none;background:#2980b9;color:#fff;padding:.4045em .809em;position:relative;line-height:50px;text-align:center;font-size:100%;*zoom:1}.wy-nav-top:after,.wy-nav-top:before{display:table;content:""}.wy-nav-top:after{clear:both}.wy-nav-top a{color:#fff;font-weight:700}.wy-nav-top img{margin-right:12px;height:45px;width:45px;background-color:#2980b9;padding:5px;border-radius:100%}.wy-nav-top i{font-size:30px;float:left;cursor:pointer;padding-top:inherit}.wy-nav-content-wrap{margin-left:300px;background:#fcfcfc;min-height:100%}.wy-nav-content{padding:1.618em 3.236em;height:100%;max-width:800px;margin:auto}.wy-body-mask{position:fixed;width:100%;height:100%;background:rgba(0,0,0,.2);display:none;z-index:499}.wy-body-mask.on{display:block}footer{color:grey}footer p{margin-bottom:12px}.rst-content footer span.commit tt,footer span.commit .rst-content tt,footer span.commit code{padding:0;font-family:SFMono-Regular,Menlo,Monaco,Consolas,Liberation Mono,Courier New,Courier,monospace;font-size:1em;background:none;border:none;color:grey}.rst-footer-buttons{*zoom:1}.rst-footer-buttons:after,.rst-footer-buttons:before{width:100%;display:table;content:""}.rst-footer-buttons:after{clear:both}.rst-breadcrumbs-buttons{margin-top:12px;*zoom:1}.rst-breadcrumbs-buttons:after,.rst-breadcrumbs-buttons:before{display:table;content:""}.rst-breadcrumbs-buttons:after{clear:both}#search-results .search li{margin-bottom:24px;border-bottom:1px solid #e1e4e5;padding-bottom:24px}#search-results .search li:first-child{border-top:1px solid #e1e4e5;padding-top:24px}#search-results .search li a{font-size:120%;margin-bottom:12px;display:inline-block}#search-results .context{color:grey;font-size:90%}.genindextable li>ul{margin-left:24px}@media screen and (max-width:768px){.wy-body-for-nav{background:#fcfcfc}.wy-nav-top{display:block}.wy-nav-side{left:-300px}.wy-nav-side.shift{width:85%;left:0}.wy-menu.wy-menu-vertical,.wy-side-nav-search,.wy-side-scroll{width:auto}.wy-nav-content-wrap{margin-left:0}.wy-nav-content-wrap .wy-nav-content{padding:1.618em}.wy-nav-content-wrap.shift{position:fixed;min-width:100%;left:85%;top:0;height:100%;overflow:hidden}}@media screen and (min-width:1100px){.wy-nav-content-wrap{background:rgba(0,0,0,.05)}.wy-nav-content{margin:0;background:#fcfcfc}}@media print{.rst-versions,.wy-nav-side,footer{display:none}.wy-nav-content-wrap{margin-left:0}}.rst-versions{position:fixed;bottom:0;left:0;width:300px;color:#fcfcfc;background:#1f1d1d;font-family:Lato,proxima-nova,Helvetica Neue,Arial,sans-serif;z-index:400}.rst-versions a{color:#2980b9;text-decoration:none}.rst-versions .rst-badge-small{display:none}.rst-versions .rst-current-version{padding:12px;background-color:#272525;display:block;text-align:right;font-size:90%;cursor:pointer;color:#27ae60;*zoom:1}.rst-versions .rst-current-version:after,.rst-versions .rst-current-version:before{display:table;content:""}.rst-versions .rst-current-version:after{clear:both}.rst-content .code-block-caption .rst-versions .rst-current-version .headerlink,.rst-content .eqno .rst-versions .rst-current-version .headerlink,.rst-content .rst-versions .rst-current-version .admonition-title,.rst-content code.download .rst-versions .rst-current-version span:first-child,.rst-content dl dt .rst-versions .rst-current-version .headerlink,.rst-content h1 .rst-versions .rst-current-version .headerlink,.rst-content h2 .rst-versions .rst-current-version .headerlink,.rst-content h3 .rst-versions .rst-current-version .headerlink,.rst-content h4 .rst-versions .rst-current-version .headerlink,.rst-content h5 .rst-versions .rst-current-version .headerlink,.rst-content h6 .rst-versions .rst-current-version .headerlink,.rst-content p .rst-versions .rst-current-version .headerlink,.rst-content table>caption .rst-versions .rst-current-version .headerlink,.rst-content tt.download .rst-versions .rst-current-version span:first-child,.rst-versions .rst-current-version .fa,.rst-versions .rst-current-version .icon,.rst-versions .rst-current-version .rst-content .admonition-title,.rst-versions .rst-current-version .rst-content .code-block-caption .headerlink,.rst-versions .rst-current-version .rst-content .eqno .headerlink,.rst-versions .rst-current-version .rst-content code.download span:first-child,.rst-versions .rst-current-version .rst-content dl dt .headerlink,.rst-versions .rst-current-version .rst-content h1 .headerlink,.rst-versions .rst-current-version .rst-content h2 .headerlink,.rst-versions .rst-current-version .rst-content h3 .headerlink,.rst-versions .rst-current-version .rst-content h4 .headerlink,.rst-versions .rst-current-version .rst-content h5 .headerlink,.rst-versions .rst-current-version .rst-content h6 .headerlink,.rst-versions .rst-current-version .rst-content p .headerlink,.rst-versions .rst-current-version .rst-content table>caption .headerlink,.rst-versions .rst-current-version .rst-content tt.download span:first-child,.rst-versions .rst-current-version .wy-menu-vertical li button.toctree-expand,.wy-menu-vertical li .rst-versions .rst-current-version button.toctree-expand{color:#fcfcfc}.rst-versions .rst-current-version .fa-book,.rst-versions .rst-current-version .icon-book{float:left}.rst-versions .rst-current-version.rst-out-of-date{background-color:#e74c3c;color:#fff}.rst-versions .rst-current-version.rst-active-old-version{background-color:#f1c40f;color:#000}.rst-versions.shift-up{height:auto;max-height:100%;overflow-y:scroll}.rst-versions.shift-up .rst-other-versions{display:block}.rst-versions .rst-other-versions{font-size:90%;padding:12px;color:grey;display:none}.rst-versions .rst-other-versions hr{display:block;height:1px;border:0;margin:20px 0;padding:0;border-top:1px solid #413d3d}.rst-versions .rst-other-versions dd{display:inline-block;margin:0}.rst-versions .rst-other-versions dd a{display:inline-block;padding:6px;color:#fcfcfc}.rst-versions.rst-badge{width:auto;bottom:20px;right:20px;left:auto;border:none;max-width:300px;max-height:90%}.rst-versions.rst-badge .fa-book,.rst-versions.rst-badge .icon-book{float:none;line-height:30px}.rst-versions.rst-badge.shift-up .rst-current-version{text-align:right}.rst-versions.rst-badge.shift-up .rst-current-version .fa-book,.rst-versions.rst-badge.shift-up .rst-current-version .icon-book{float:left}.rst-versions.rst-badge>.rst-current-version{width:auto;height:30px;line-height:30px;padding:0 6px;display:block;text-align:center}@media screen and (max-width:768px){.rst-versions{width:85%;display:none}.rst-versions.shift{display:block}}.rst-content .toctree-wrapper>p.caption,.rst-content h1,.rst-content h2,.rst-content h3,.rst-content h4,.rst-content h5,.rst-content h6{margin-bottom:24px}.rst-content img{max-width:100%;height:auto}.rst-content div.figure,.rst-content figure{margin-bottom:24px}.rst-content div.figure .caption-text,.rst-content figure .caption-text{font-style:italic}.rst-content div.figure p:last-child.caption,.rst-content figure p:last-child.caption{margin-bottom:0}.rst-content div.figure.align-center,.rst-content figure.align-center{text-align:center}.rst-content .section>a>img,.rst-content .section>img,.rst-content section>a>img,.rst-content section>img{margin-bottom:24px}.rst-content abbr[title]{text-decoration:none}.rst-content.style-external-links a.reference.external:after{font-family:FontAwesome;content:"\f08e";color:#b3b3b3;vertical-align:super;font-size:60%;margin:0 .2em}.rst-content blockquote{margin-left:24px;line-height:24px;margin-bottom:24px}.rst-content pre.literal-block{white-space:pre;margin:0;padding:12px;font-family:SFMono-Regular,Menlo,Monaco,Consolas,Liberation Mono,Courier New,Courier,monospace;display:block;overflow:auto}.rst-content div[class^=highlight],.rst-content pre.literal-block{border:1px solid #e1e4e5;overflow-x:auto;margin:1px 0 24px}.rst-content div[class^=highlight] div[class^=highlight],.rst-content pre.literal-block div[class^=highlight]{padding:0;border:none;margin:0}.rst-content div[class^=highlight] td.code{width:100%}.rst-content .linenodiv pre{border-right:1px solid #e6e9ea;margin:0;padding:12px;font-family:SFMono-Regular,Menlo,Monaco,Consolas,Liberation Mono,Courier New,Courier,monospace;user-select:none;pointer-events:none}.rst-content div[class^=highlight] pre{white-space:pre;margin:0;padding:12px;display:block;overflow:auto}.rst-content div[class^=highlight] pre .hll{display:block;margin:0 -12px;padding:0 12px}.rst-content .linenodiv pre,.rst-content div[class^=highlight] pre,.rst-content pre.literal-block{font-family:SFMono-Regular,Menlo,Monaco,Consolas,Liberation Mono,Courier New,Courier,monospace;font-size:12px;line-height:1.4}.rst-content div.highlight .gp,.rst-content div.highlight span.linenos{user-select:none;pointer-events:none}.rst-content div.highlight span.linenos{display:inline-block;padding-left:0;padding-right:12px;margin-right:12px;border-right:1px solid #e6e9ea}.rst-content .code-block-caption{font-style:italic;font-size:85%;line-height:1;padding:1em 0;text-align:center}@media print{.rst-content .codeblock,.rst-content div[class^=highlight],.rst-content div[class^=highlight] pre{white-space:pre-wrap}}.rst-content .admonition,.rst-content .admonition-todo,.rst-content .attention,.rst-content .caution,.rst-content .danger,.rst-content .error,.rst-content .hint,.rst-content .important,.rst-content .note,.rst-content .seealso,.rst-content .tip,.rst-content .warning{clear:both}.rst-content .admonition-todo .last,.rst-content .admonition-todo>:last-child,.rst-content .admonition .last,.rst-content .admonition>:last-child,.rst-content .attention .last,.rst-content .attention>:last-child,.rst-content .caution .last,.rst-content .caution>:last-child,.rst-content .danger .last,.rst-content .danger>:last-child,.rst-content .error .last,.rst-content .error>:last-child,.rst-content .hint .last,.rst-content .hint>:last-child,.rst-content .important .last,.rst-content .important>:last-child,.rst-content .note .last,.rst-content .note>:last-child,.rst-content .seealso .last,.rst-content .seealso>:last-child,.rst-content .tip .last,.rst-content .tip>:last-child,.rst-content .warning .last,.rst-content .warning>:last-child{margin-bottom:0}.rst-content .admonition-title:before{margin-right:4px}.rst-content .admonition table{border-color:rgba(0,0,0,.1)}.rst-content .admonition table td,.rst-content .admonition table th{background:transparent!important;border-color:rgba(0,0,0,.1)!important}.rst-content .section ol.loweralpha,.rst-content .section ol.loweralpha>li,.rst-content .toctree-wrapper ol.loweralpha,.rst-content .toctree-wrapper ol.loweralpha>li,.rst-content section ol.loweralpha,.rst-content section ol.loweralpha>li{list-style:lower-alpha}.rst-content .section ol.upperalpha,.rst-content .section ol.upperalpha>li,.rst-content .toctree-wrapper ol.upperalpha,.rst-content .toctree-wrapper ol.upperalpha>li,.rst-content section ol.upperalpha,.rst-content section ol.upperalpha>li{list-style:upper-alpha}.rst-content .section ol li>*,.rst-content .section ul li>*,.rst-content .toctree-wrapper ol li>*,.rst-content .toctree-wrapper ul li>*,.rst-content section ol li>*,.rst-content section ul li>*{margin-top:12px;margin-bottom:12px}.rst-content .section ol li>:first-child,.rst-content .section ul li>:first-child,.rst-content .toctree-wrapper ol li>:first-child,.rst-content .toctree-wrapper ul li>:first-child,.rst-content section ol li>:first-child,.rst-content section ul li>:first-child{margin-top:0}.rst-content .section ol li>p,.rst-content .section ol li>p:last-child,.rst-content .section ul li>p,.rst-content .section ul li>p:last-child,.rst-content .toctree-wrapper ol li>p,.rst-content .toctree-wrapper ol li>p:last-child,.rst-content .toctree-wrapper ul li>p,.rst-content .toctree-wrapper ul li>p:last-child,.rst-content section ol li>p,.rst-content section ol li>p:last-child,.rst-content section ul li>p,.rst-content section ul li>p:last-child{margin-bottom:12px}.rst-content .section ol li>p:only-child,.rst-content .section ol li>p:only-child:last-child,.rst-content .section ul li>p:only-child,.rst-content .section ul li>p:only-child:last-child,.rst-content .toctree-wrapper ol li>p:only-child,.rst-content .toctree-wrapper ol li>p:only-child:last-child,.rst-content .toctree-wrapper ul li>p:only-child,.rst-content .toctree-wrapper ul li>p:only-child:last-child,.rst-content section ol li>p:only-child,.rst-content section ol li>p:only-child:last-child,.rst-content section ul li>p:only-child,.rst-content section ul li>p:only-child:last-child{margin-bottom:0}.rst-content .section ol li>ol,.rst-content .section ol li>ul,.rst-content .section ul li>ol,.rst-content .section ul li>ul,.rst-content .toctree-wrapper ol li>ol,.rst-content .toctree-wrapper ol li>ul,.rst-content .toctree-wrapper ul li>ol,.rst-content .toctree-wrapper ul li>ul,.rst-content section ol li>ol,.rst-content section ol li>ul,.rst-content section ul li>ol,.rst-content section ul li>ul{margin-bottom:12px}.rst-content .section ol.simple li>*,.rst-content .section ol.simple li ol,.rst-content .section ol.simple li ul,.rst-content .section ul.simple li>*,.rst-content .section ul.simple li ol,.rst-content .section ul.simple li ul,.rst-content .toctree-wrapper ol.simple li>*,.rst-content .toctree-wrapper ol.simple li ol,.rst-content .toctree-wrapper ol.simple li ul,.rst-content .toctree-wrapper ul.simple li>*,.rst-content .toctree-wrapper ul.simple li ol,.rst-content .toctree-wrapper ul.simple li ul,.rst-content section ol.simple li>*,.rst-content section ol.simple li ol,.rst-content section ol.simple li ul,.rst-content section ul.simple li>*,.rst-content section ul.simple li ol,.rst-content section ul.simple li ul{margin-top:0;margin-bottom:0}.rst-content .line-block{margin-left:0;margin-bottom:24px;line-height:24px}.rst-content .line-block .line-block{margin-left:24px;margin-bottom:0}.rst-content .topic-title{font-weight:700;margin-bottom:12px}.rst-content .toc-backref{color:#404040}.rst-content .align-right{float:right;margin:0 0 24px 24px}.rst-content .align-left{float:left;margin:0 24px 24px 0}.rst-content .align-center{margin:auto}.rst-content .align-center:not(table){display:block}.rst-content .code-block-caption .headerlink,.rst-content .eqno .headerlink,.rst-content .toctree-wrapper>p.caption .headerlink,.rst-content dl dt .headerlink,.rst-content h1 .headerlink,.rst-content h2 .headerlink,.rst-content h3 .headerlink,.rst-content h4 .headerlink,.rst-content h5 .headerlink,.rst-content h6 .headerlink,.rst-content p.caption .headerlink,.rst-content p .headerlink,.rst-content table>caption .headerlink{opacity:0;font-size:14px;font-family:FontAwesome;margin-left:.5em}.rst-content .code-block-caption .headerlink:focus,.rst-content .code-block-caption:hover .headerlink,.rst-content .eqno .headerlink:focus,.rst-content .eqno:hover .headerlink,.rst-content .toctree-wrapper>p.caption .headerlink:focus,.rst-content .toctree-wrapper>p.caption:hover .headerlink,.rst-content dl dt .headerlink:focus,.rst-content dl dt:hover .headerlink,.rst-content h1 .headerlink:focus,.rst-content h1:hover .headerlink,.rst-content h2 .headerlink:focus,.rst-content h2:hover .headerlink,.rst-content h3 .headerlink:focus,.rst-content h3:hover .headerlink,.rst-content h4 .headerlink:focus,.rst-content h4:hover .headerlink,.rst-content h5 .headerlink:focus,.rst-content h5:hover .headerlink,.rst-content h6 .headerlink:focus,.rst-content h6:hover .headerlink,.rst-content p.caption .headerlink:focus,.rst-content p.caption:hover .headerlink,.rst-content p .headerlink:focus,.rst-content p:hover .headerlink,.rst-content table>caption .headerlink:focus,.rst-content table>caption:hover .headerlink{opacity:1}.rst-content p a{overflow-wrap:anywhere}.rst-content .wy-table td p,.rst-content .wy-table td ul,.rst-content .wy-table th p,.rst-content .wy-table th ul,.rst-content table.docutils td p,.rst-content table.docutils td ul,.rst-content table.docutils th p,.rst-content table.docutils th ul,.rst-content table.field-list td p,.rst-content table.field-list td ul,.rst-content table.field-list th p,.rst-content table.field-list th ul{font-size:inherit}.rst-content .btn:focus{outline:2px solid}.rst-content table>caption .headerlink:after{font-size:12px}.rst-content .centered{text-align:center}.rst-content .sidebar{float:right;width:40%;display:block;margin:0 0 24px 24px;padding:24px;background:#f3f6f6;border:1px solid #e1e4e5}.rst-content .sidebar dl,.rst-content .sidebar p,.rst-content .sidebar ul{font-size:90%}.rst-content .sidebar .last,.rst-content .sidebar>:last-child{margin-bottom:0}.rst-content .sidebar .sidebar-title{display:block;font-family:Roboto Slab,ff-tisa-web-pro,Georgia,Arial,sans-serif;font-weight:700;background:#e1e4e5;padding:6px 12px;margin:-24px -24px 24px;font-size:100%}.rst-content .highlighted{background:#f1c40f;box-shadow:0 0 0 2px #f1c40f;display:inline;font-weight:700}.rst-content .citation-reference,.rst-content .footnote-reference{vertical-align:baseline;position:relative;top:-.4em;line-height:0;font-size:90%}.rst-content .citation-reference>span.fn-bracket,.rst-content .footnote-reference>span.fn-bracket{display:none}.rst-content .hlist{width:100%}.rst-content dl dt span.classifier:before{content:" : "}.rst-content dl dt span.classifier-delimiter{display:none!important}html.writer-html4 .rst-content table.docutils.citation,html.writer-html4 .rst-content table.docutils.footnote{background:none;border:none}html.writer-html4 .rst-content table.docutils.citation td,html.writer-html4 .rst-content table.docutils.citation tr,html.writer-html4 .rst-content table.docutils.footnote td,html.writer-html4 .rst-content table.docutils.footnote tr{border:none;background-color:transparent!important;white-space:normal}html.writer-html4 .rst-content table.docutils.citation td.label,html.writer-html4 .rst-content table.docutils.footnote td.label{padding-left:0;padding-right:0;vertical-align:top}html.writer-html5 .rst-content dl.citation,html.writer-html5 .rst-content dl.field-list,html.writer-html5 .rst-content dl.footnote{display:grid;grid-template-columns:auto minmax(80%,95%)}html.writer-html5 .rst-content dl.citation>dt,html.writer-html5 .rst-content dl.field-list>dt,html.writer-html5 .rst-content dl.footnote>dt{display:inline-grid;grid-template-columns:max-content auto}html.writer-html5 .rst-content aside.citation,html.writer-html5 .rst-content aside.footnote,html.writer-html5 .rst-content div.citation{display:grid;grid-template-columns:auto auto minmax(.65rem,auto) minmax(40%,95%)}html.writer-html5 .rst-content aside.citation>span.label,html.writer-html5 .rst-content aside.footnote>span.label,html.writer-html5 .rst-content div.citation>span.label{grid-column-start:1;grid-column-end:2}html.writer-html5 .rst-content aside.citation>span.backrefs,html.writer-html5 .rst-content aside.footnote>span.backrefs,html.writer-html5 .rst-content div.citation>span.backrefs{grid-column-start:2;grid-column-end:3;grid-row-start:1;grid-row-end:3}html.writer-html5 .rst-content aside.citation>p,html.writer-html5 .rst-content aside.footnote>p,html.writer-html5 .rst-content div.citation>p{grid-column-start:4;grid-column-end:5}html.writer-html5 .rst-content dl.citation,html.writer-html5 .rst-content dl.field-list,html.writer-html5 .rst-content dl.footnote{margin-bottom:24px}html.writer-html5 .rst-content dl.citation>dt,html.writer-html5 .rst-content dl.field-list>dt,html.writer-html5 .rst-content dl.footnote>dt{padding-left:1rem}html.writer-html5 .rst-content dl.citation>dd,html.writer-html5 .rst-content dl.citation>dt,html.writer-html5 .rst-content dl.field-list>dd,html.writer-html5 .rst-content dl.field-list>dt,html.writer-html5 .rst-content dl.footnote>dd,html.writer-html5 .rst-content dl.footnote>dt{margin-bottom:0}html.writer-html5 .rst-content dl.citation,html.writer-html5 .rst-content dl.footnote{font-size:.9rem}html.writer-html5 .rst-content dl.citation>dt,html.writer-html5 .rst-content dl.footnote>dt{margin:0 .5rem .5rem 0;line-height:1.2rem;word-break:break-all;font-weight:400}html.writer-html5 .rst-content dl.citation>dt>span.brackets:before,html.writer-html5 .rst-content dl.footnote>dt>span.brackets:before{content:"["}html.writer-html5 .rst-content dl.citation>dt>span.brackets:after,html.writer-html5 .rst-content dl.footnote>dt>span.brackets:after{content:"]"}html.writer-html5 .rst-content dl.citation>dt>span.fn-backref,html.writer-html5 .rst-content dl.footnote>dt>span.fn-backref{text-align:left;font-style:italic;margin-left:.65rem;word-break:break-word;word-spacing:-.1rem;max-width:5rem}html.writer-html5 .rst-content dl.citation>dt>span.fn-backref>a,html.writer-html5 .rst-content dl.footnote>dt>span.fn-backref>a{word-break:keep-all}html.writer-html5 .rst-content dl.citation>dt>span.fn-backref>a:not(:first-child):before,html.writer-html5 .rst-content dl.footnote>dt>span.fn-backref>a:not(:first-child):before{content:" "}html.writer-html5 .rst-content dl.citation>dd,html.writer-html5 .rst-content dl.footnote>dd{margin:0 0 .5rem;line-height:1.2rem}html.writer-html5 .rst-content dl.citation>dd p,html.writer-html5 .rst-content dl.footnote>dd p{font-size:.9rem}html.writer-html5 .rst-content aside.citation,html.writer-html5 .rst-content aside.footnote,html.writer-html5 .rst-content div.citation{padding-left:1rem;padding-right:1rem;font-size:.9rem;line-height:1.2rem}html.writer-html5 .rst-content aside.citation p,html.writer-html5 .rst-content aside.footnote p,html.writer-html5 .rst-content div.citation p{font-size:.9rem;line-height:1.2rem;margin-bottom:12px}html.writer-html5 .rst-content aside.citation span.backrefs,html.writer-html5 .rst-content aside.footnote span.backrefs,html.writer-html5 .rst-content div.citation span.backrefs{text-align:left;font-style:italic;margin-left:.65rem;word-break:break-word;word-spacing:-.1rem;max-width:5rem}html.writer-html5 .rst-content aside.citation span.backrefs>a,html.writer-html5 .rst-content aside.footnote span.backrefs>a,html.writer-html5 .rst-content div.citation span.backrefs>a{word-break:keep-all}html.writer-html5 .rst-content aside.citation span.backrefs>a:not(:first-child):before,html.writer-html5 .rst-content aside.footnote span.backrefs>a:not(:first-child):before,html.writer-html5 .rst-content div.citation span.backrefs>a:not(:first-child):before{content:" "}html.writer-html5 .rst-content aside.citation span.label,html.writer-html5 .rst-content aside.footnote span.label,html.writer-html5 .rst-content div.citation span.label{line-height:1.2rem}html.writer-html5 .rst-content aside.citation-list,html.writer-html5 .rst-content aside.footnote-list,html.writer-html5 .rst-content div.citation-list{margin-bottom:24px}html.writer-html5 .rst-content dl.option-list kbd{font-size:.9rem}.rst-content table.docutils.footnote,html.writer-html4 .rst-content table.docutils.citation,html.writer-html5 .rst-content aside.footnote,html.writer-html5 .rst-content aside.footnote-list aside.footnote,html.writer-html5 .rst-content div.citation-list>div.citation,html.writer-html5 .rst-content dl.citation,html.writer-html5 .rst-content dl.footnote{color:grey}.rst-content table.docutils.footnote code,.rst-content table.docutils.footnote tt,html.writer-html4 .rst-content table.docutils.citation code,html.writer-html4 .rst-content table.docutils.citation tt,html.writer-html5 .rst-content aside.footnote-list aside.footnote code,html.writer-html5 .rst-content aside.footnote-list aside.footnote tt,html.writer-html5 .rst-content aside.footnote code,html.writer-html5 .rst-content aside.footnote tt,html.writer-html5 .rst-content div.citation-list>div.citation code,html.writer-html5 .rst-content div.citation-list>div.citation tt,html.writer-html5 .rst-content dl.citation code,html.writer-html5 .rst-content dl.citation tt,html.writer-html5 .rst-content dl.footnote code,html.writer-html5 .rst-content dl.footnote tt{color:#555}.rst-content .wy-table-responsive.citation,.rst-content .wy-table-responsive.footnote{margin-bottom:0}.rst-content .wy-table-responsive.citation+:not(.citation),.rst-content .wy-table-responsive.footnote+:not(.footnote){margin-top:24px}.rst-content .wy-table-responsive.citation:last-child,.rst-content .wy-table-responsive.footnote:last-child{margin-bottom:24px}.rst-content table.docutils th{border-color:#e1e4e5}html.writer-html5 .rst-content table.docutils th{border:1px solid #e1e4e5}html.writer-html5 .rst-content table.docutils td>p,html.writer-html5 .rst-content table.docutils th>p{line-height:1rem;margin-bottom:0;font-size:.9rem}.rst-content table.docutils td .last,.rst-content table.docutils td .last>:last-child{margin-bottom:0}.rst-content table.field-list,.rst-content table.field-list td{border:none}.rst-content table.field-list td p{line-height:inherit}.rst-content table.field-list td>strong{display:inline-block}.rst-content table.field-list .field-name{padding-right:10px;text-align:left;white-space:nowrap}.rst-content table.field-list .field-body{text-align:left}.rst-content code,.rst-content tt{color:#000;font-family:SFMono-Regular,Menlo,Monaco,Consolas,Liberation Mono,Courier New,Courier,monospace;padding:2px 5px}.rst-content code big,.rst-content code em,.rst-content tt big,.rst-content tt em{font-size:100%!important;line-height:normal}.rst-content code.literal,.rst-content tt.literal{color:#e74c3c;white-space:normal}.rst-content code.xref,.rst-content tt.xref,a .rst-content code,a .rst-content tt{font-weight:700;color:#404040;overflow-wrap:normal}.rst-content kbd,.rst-content pre,.rst-content samp{font-family:SFMono-Regular,Menlo,Monaco,Consolas,Liberation Mono,Courier New,Courier,monospace}.rst-content a code,.rst-content a tt{color:#2980b9}.rst-content dl{margin-bottom:24px}.rst-content dl dt{font-weight:700;margin-bottom:12px}.rst-content dl ol,.rst-content dl p,.rst-content dl table,.rst-content dl ul{margin-bottom:12px}.rst-content dl dd{margin:0 0 12px 24px;line-height:24px}.rst-content dl dd>ol:last-child,.rst-content dl dd>p:last-child,.rst-content dl dd>table:last-child,.rst-content dl dd>ul:last-child{margin-bottom:0}html.writer-html4 .rst-content dl:not(.docutils),html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple){margin-bottom:24px}html.writer-html4 .rst-content dl:not(.docutils)>dt,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple)>dt{display:table;margin:6px 0;font-size:90%;line-height:normal;background:#e7f2fa;color:#2980b9;border-top:3px solid #6ab0de;padding:6px;position:relative}html.writer-html4 .rst-content dl:not(.docutils)>dt:before,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple)>dt:before{color:#6ab0de}html.writer-html4 .rst-content dl:not(.docutils)>dt .headerlink,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple)>dt .headerlink{color:#404040;font-size:100%!important}html.writer-html4 .rst-content dl:not(.docutils) dl:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple)>dt,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) dl:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple)>dt{margin-bottom:6px;border:none;border-left:3px solid #ccc;background:#f0f0f0;color:#555}html.writer-html4 .rst-content dl:not(.docutils) dl:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple)>dt .headerlink,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) dl:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple)>dt .headerlink{color:#404040;font-size:100%!important}html.writer-html4 .rst-content dl:not(.docutils)>dt:first-child,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple)>dt:first-child{margin-top:0}html.writer-html4 .rst-content dl:not(.docutils) code.descclassname,html.writer-html4 .rst-content dl:not(.docutils) code.descname,html.writer-html4 .rst-content dl:not(.docutils) tt.descclassname,html.writer-html4 .rst-content dl:not(.docutils) tt.descname,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) code.descclassname,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) code.descname,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) tt.descclassname,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) tt.descname{background-color:transparent;border:none;padding:0;font-size:100%!important}html.writer-html4 .rst-content dl:not(.docutils) code.descname,html.writer-html4 .rst-content dl:not(.docutils) tt.descname,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) code.descname,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) tt.descname{font-weight:700}html.writer-html4 .rst-content dl:not(.docutils) .optional,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) .optional{display:inline-block;padding:0 4px;color:#000;font-weight:700}html.writer-html4 .rst-content dl:not(.docutils) .property,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) .property{display:inline-block;padding-right:8px;max-width:100%}html.writer-html4 .rst-content dl:not(.docutils) .k,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) .k{font-style:italic}html.writer-html4 .rst-content dl:not(.docutils) .descclassname,html.writer-html4 .rst-content dl:not(.docutils) .descname,html.writer-html4 .rst-content dl:not(.docutils) .sig-name,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) .descclassname,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) .descname,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) .sig-name{font-family:SFMono-Regular,Menlo,Monaco,Consolas,Liberation Mono,Courier New,Courier,monospace;color:#000}.rst-content .viewcode-back,.rst-content .viewcode-link{display:inline-block;color:#27ae60;font-size:80%;padding-left:24px}.rst-content .viewcode-back{display:block;float:right}.rst-content p.rubric{margin-bottom:12px;font-weight:700}.rst-content code.download,.rst-content tt.download{background:inherit;padding:inherit;font-weight:400;font-family:inherit;font-size:inherit;color:inherit;border:inherit;white-space:inherit}.rst-content code.download span:first-child,.rst-content tt.download span:first-child{-webkit-font-smoothing:subpixel-antialiased}.rst-content code.download span:first-child:before,.rst-content tt.download span:first-child:before{margin-right:4px}.rst-content .guilabel,.rst-content .menuselection{font-size:80%;font-weight:700;border-radius:4px;padding:2.4px 6px;margin:auto 2px}.rst-content .guilabel,.rst-content .menuselection{border:1px solid #7fbbe3;background:#e7f2fa}.rst-content :not(dl.option-list)>:not(dt):not(kbd):not(.kbd)>.kbd,.rst-content :not(dl.option-list)>:not(dt):not(kbd):not(.kbd)>kbd{color:inherit;font-size:80%;background-color:#fff;border:1px solid #a6a6a6;border-radius:4px;box-shadow:0 2px grey;padding:2.4px 6px;margin:auto 0}.rst-content .versionmodified{font-style:italic}@media screen and (max-width:480px){.rst-content .sidebar{width:100%}}span[id*=MathJax-Span]{color:#404040}.math{text-align:center}@font-face{font-family:Lato;src:url(fonts/lato-normal.woff2?bd03a2cc277bbbc338d464e679fe9942) format("woff2"),url(fonts/lato-normal.woff?27bd77b9162d388cb8d4c4217c7c5e2a) format("woff");font-weight:400;font-style:normal;font-display:block}@font-face{font-family:Lato;src:url(fonts/lato-bold.woff2?cccb897485813c7c256901dbca54ecf2) format("woff2"),url(fonts/lato-bold.woff?d878b6c29b10beca227e9eef4246111b) format("woff");font-weight:700;font-style:normal;font-display:block}@font-face{font-family:Lato;src:url(fonts/lato-bold-italic.woff2?0b6bb6725576b072c5d0b02ecdd1900d) format("woff2"),url(fonts/lato-bold-italic.woff?9c7e4e9eb485b4a121c760e61bc3707c) format("woff");font-weight:700;font-style:italic;font-display:block}@font-face{font-family:Lato;src:url(fonts/lato-normal-italic.woff2?4eb103b4d12be57cb1d040ed5e162e9d) format("woff2"),url(fonts/lato-normal-italic.woff?f28f2d6482446544ef1ea1ccc6dd5892) format("woff");font-weight:400;font-style:italic;font-display:block}@font-face{font-family:Roboto Slab;font-style:normal;font-weight:400;src:url(fonts/Roboto-Slab-Regular.woff2?7abf5b8d04d26a2cafea937019bca958) format("woff2"),url(fonts/Roboto-Slab-Regular.woff?c1be9284088d487c5e3ff0a10a92e58c) format("woff");font-display:block}@font-face{font-family:Roboto Slab;font-style:normal;font-weight:700;src:url(fonts/Roboto-Slab-Bold.woff2?9984f4a9bda09be08e83f2506954adbe) format("woff2"),url(fonts/Roboto-Slab-Bold.woff?bed5564a116b05148e3b3bea6fb1162a) format("woff");font-display:block} \ No newline at end of file diff --git a/docs/build/html/_static/doctools.js b/docs/build/html/_static/doctools.js new file mode 100644 index 000000000..c3db08d1c --- /dev/null +++ b/docs/build/html/_static/doctools.js @@ -0,0 +1,264 @@ +/* + * doctools.js + * ~~~~~~~~~~~ + * + * Base JavaScript utilities for all Sphinx HTML documentation. + * + * :copyright: Copyright 2007-2022 by the Sphinx team, see AUTHORS. + * :license: BSD, see LICENSE for details. + * + */ +"use strict"; + +const _ready = (callback) => { + if (document.readyState !== "loading") { + callback(); + } else { + document.addEventListener("DOMContentLoaded", callback); + } +}; + +/** + * highlight a given string on a node by wrapping it in + * span elements with the given class name. + */ +const _highlight = (node, addItems, text, className) => { + if (node.nodeType === Node.TEXT_NODE) { + const val = node.nodeValue; + const parent = node.parentNode; + const pos = val.toLowerCase().indexOf(text); + if ( + pos >= 0 && + !parent.classList.contains(className) && + !parent.classList.contains("nohighlight") + ) { + let span; + + const closestNode = parent.closest("body, svg, foreignObject"); + const isInSVG = closestNode && closestNode.matches("svg"); + if (isInSVG) { + span = document.createElementNS("http://www.w3.org/2000/svg", "tspan"); + } else { + span = document.createElement("span"); + span.classList.add(className); + } + + span.appendChild(document.createTextNode(val.substr(pos, text.length))); + parent.insertBefore( + span, + parent.insertBefore( + document.createTextNode(val.substr(pos + text.length)), + node.nextSibling + ) + ); + node.nodeValue = val.substr(0, pos); + + if (isInSVG) { + const rect = document.createElementNS( + "http://www.w3.org/2000/svg", + "rect" + ); + const bbox = parent.getBBox(); + rect.x.baseVal.value = bbox.x; + rect.y.baseVal.value = bbox.y; + rect.width.baseVal.value = bbox.width; + rect.height.baseVal.value = bbox.height; + rect.setAttribute("class", className); + addItems.push({ parent: parent, target: rect }); + } + } + } else if (node.matches && !node.matches("button, select, textarea")) { + node.childNodes.forEach((el) => _highlight(el, addItems, text, className)); + } +}; +const _highlightText = (thisNode, text, className) => { + let addItems = []; + _highlight(thisNode, addItems, text, className); + addItems.forEach((obj) => + obj.parent.insertAdjacentElement("beforebegin", obj.target) + ); +}; + +/** + * Small JavaScript module for the documentation. + */ +const Documentation = { + init: () => { + Documentation.highlightSearchWords(); + Documentation.initDomainIndexTable(); + Documentation.initOnKeyListeners(); + }, + + /** + * i18n support + */ + TRANSLATIONS: {}, + PLURAL_EXPR: (n) => (n === 1 ? 0 : 1), + LOCALE: "unknown", + + // gettext and ngettext don't access this so that the functions + // can safely bound to a different name (_ = Documentation.gettext) + gettext: (string) => { + const translated = Documentation.TRANSLATIONS[string]; + switch (typeof translated) { + case "undefined": + return string; // no translation + case "string": + return translated; // translation exists + default: + return translated[0]; // (singular, plural) translation tuple exists + } + }, + + ngettext: (singular, plural, n) => { + const translated = Documentation.TRANSLATIONS[singular]; + if (typeof translated !== "undefined") + return translated[Documentation.PLURAL_EXPR(n)]; + return n === 1 ? singular : plural; + }, + + addTranslations: (catalog) => { + Object.assign(Documentation.TRANSLATIONS, catalog.messages); + Documentation.PLURAL_EXPR = new Function( + "n", + `return (${catalog.plural_expr})` + ); + Documentation.LOCALE = catalog.locale; + }, + + /** + * highlight the search words provided in the url in the text + */ + highlightSearchWords: () => { + const highlight = + new URLSearchParams(window.location.search).get("highlight") || ""; + const terms = highlight.toLowerCase().split(/\s+/).filter(x => x); + if (terms.length === 0) return; // nothing to do + + // There should never be more than one element matching "div.body" + const divBody = document.querySelectorAll("div.body"); + const body = divBody.length ? divBody[0] : document.querySelector("body"); + window.setTimeout(() => { + terms.forEach((term) => _highlightText(body, term, "highlighted")); + }, 10); + + const searchBox = document.getElementById("searchbox"); + if (searchBox === null) return; + searchBox.appendChild( + document + .createRange() + .createContextualFragment( + '

' + + '' + + Documentation.gettext("Hide Search Matches") + + "

" + ) + ); + }, + + /** + * helper function to hide the search marks again + */ + hideSearchWords: () => { + document + .querySelectorAll("#searchbox .highlight-link") + .forEach((el) => el.remove()); + document + .querySelectorAll("span.highlighted") + .forEach((el) => el.classList.remove("highlighted")); + const url = new URL(window.location); + url.searchParams.delete("highlight"); + window.history.replaceState({}, "", url); + }, + + /** + * helper function to focus on search bar + */ + focusSearchBar: () => { + document.querySelectorAll("input[name=q]")[0]?.focus(); + }, + + /** + * Initialise the domain index toggle buttons + */ + initDomainIndexTable: () => { + const toggler = (el) => { + const idNumber = el.id.substr(7); + const toggledRows = document.querySelectorAll(`tr.cg-${idNumber}`); + if (el.src.substr(-9) === "minus.png") { + el.src = `${el.src.substr(0, el.src.length - 9)}plus.png`; + toggledRows.forEach((el) => (el.style.display = "none")); + } else { + el.src = `${el.src.substr(0, el.src.length - 8)}minus.png`; + toggledRows.forEach((el) => (el.style.display = "")); + } + }; + + const togglerElements = document.querySelectorAll("img.toggler"); + togglerElements.forEach((el) => + el.addEventListener("click", (event) => toggler(event.currentTarget)) + ); + togglerElements.forEach((el) => (el.style.display = "")); + if (DOCUMENTATION_OPTIONS.COLLAPSE_INDEX) togglerElements.forEach(toggler); + }, + + initOnKeyListeners: () => { + // only install a listener if it is really needed + if ( + !DOCUMENTATION_OPTIONS.NAVIGATION_WITH_KEYS && + !DOCUMENTATION_OPTIONS.ENABLE_SEARCH_SHORTCUTS + ) + return; + + const blacklistedElements = new Set([ + "TEXTAREA", + "INPUT", + "SELECT", + "BUTTON", + ]); + document.addEventListener("keydown", (event) => { + if (blacklistedElements.has(document.activeElement.tagName)) return; // bail for input elements + if (event.altKey || event.ctrlKey || event.metaKey) return; // bail with special keys + + if (!event.shiftKey) { + switch (event.key) { + case "ArrowLeft": + if (!DOCUMENTATION_OPTIONS.NAVIGATION_WITH_KEYS) break; + + const prevLink = document.querySelector('link[rel="prev"]'); + if (prevLink && prevLink.href) { + window.location.href = prevLink.href; + event.preventDefault(); + } + break; + case "ArrowRight": + if (!DOCUMENTATION_OPTIONS.NAVIGATION_WITH_KEYS) break; + + const nextLink = document.querySelector('link[rel="next"]'); + if (nextLink && nextLink.href) { + window.location.href = nextLink.href; + event.preventDefault(); + } + break; + case "Escape": + if (!DOCUMENTATION_OPTIONS.ENABLE_SEARCH_SHORTCUTS) break; + Documentation.hideSearchWords(); + event.preventDefault(); + } + } + + // some keyboard layouts may need Shift to get / + switch (event.key) { + case "/": + if (!DOCUMENTATION_OPTIONS.ENABLE_SEARCH_SHORTCUTS) break; + Documentation.focusSearchBar(); + event.preventDefault(); + } + }); + }, +}; + +// quick alias for translations +const _ = Documentation.gettext; + +_ready(Documentation.init); diff --git a/docs/build/html/_static/documentation_options.js b/docs/build/html/_static/documentation_options.js new file mode 100644 index 000000000..b8457d918 --- /dev/null +++ b/docs/build/html/_static/documentation_options.js @@ -0,0 +1,14 @@ +var DOCUMENTATION_OPTIONS = { + URL_ROOT: document.getElementById("documentation_options").getAttribute('data-url_root'), + VERSION: '0.0.0', + LANGUAGE: 'en', + COLLAPSE_INDEX: false, + BUILDER: 'html', + FILE_SUFFIX: '.html', + LINK_SUFFIX: '.html', + HAS_SOURCE: true, + SOURCELINK_SUFFIX: '.txt', + NAVIGATION_WITH_KEYS: false, + SHOW_SEARCH_SUMMARY: true, + ENABLE_SEARCH_SHORTCUTS: false, +}; \ No newline at end of file diff --git a/docs/build/html/_static/file.png b/docs/build/html/_static/file.png new file mode 100644 index 000000000..a858a410e Binary files /dev/null and b/docs/build/html/_static/file.png differ diff --git a/docs/build/html/_static/jquery-3.6.0.js b/docs/build/html/_static/jquery-3.6.0.js new file mode 100644 index 000000000..fc6c299b7 --- /dev/null +++ b/docs/build/html/_static/jquery-3.6.0.js @@ -0,0 +1,10881 @@ +/*! + * jQuery JavaScript Library v3.6.0 + * https://jquery.com/ + * + * Includes Sizzle.js + * https://sizzlejs.com/ + * + * Copyright OpenJS Foundation and other contributors + * Released under the MIT license + * https://jquery.org/license + * + * Date: 2021-03-02T17:08Z + */ +( function( global, factory ) { + + "use strict"; + + if ( typeof module === "object" && typeof module.exports === "object" ) { + + // For CommonJS and CommonJS-like environments where a proper `window` + // is present, execute the factory and get jQuery. + // For environments that do not have a `window` with a `document` + // (such as Node.js), expose a factory as module.exports. + // This accentuates the need for the creation of a real `window`. + // e.g. var jQuery = require("jquery")(window); + // See ticket #14549 for more info. + module.exports = global.document ? + factory( global, true ) : + function( w ) { + if ( !w.document ) { + throw new Error( "jQuery requires a window with a document" ); + } + return factory( w ); + }; + } else { + factory( global ); + } + +// Pass this if window is not defined yet +} )( typeof window !== "undefined" ? window : this, function( window, noGlobal ) { + +// Edge <= 12 - 13+, Firefox <=18 - 45+, IE 10 - 11, Safari 5.1 - 9+, iOS 6 - 9.1 +// throw exceptions when non-strict code (e.g., ASP.NET 4.5) accesses strict mode +// arguments.callee.caller (trac-13335). But as of jQuery 3.0 (2016), strict mode should be common +// enough that all such attempts are guarded in a try block. +"use strict"; + +var arr = []; + +var getProto = Object.getPrototypeOf; + +var slice = arr.slice; + +var flat = arr.flat ? function( array ) { + return arr.flat.call( array ); +} : function( array ) { + return arr.concat.apply( [], array ); +}; + + +var push = arr.push; + +var indexOf = arr.indexOf; + +var class2type = {}; + +var toString = class2type.toString; + +var hasOwn = class2type.hasOwnProperty; + +var fnToString = hasOwn.toString; + +var ObjectFunctionString = fnToString.call( Object ); + +var support = {}; + +var isFunction = function isFunction( obj ) { + + // Support: Chrome <=57, Firefox <=52 + // In some browsers, typeof returns "function" for HTML elements + // (i.e., `typeof document.createElement( "object" ) === "function"`). + // We don't want to classify *any* DOM node as a function. + // Support: QtWeb <=3.8.5, WebKit <=534.34, wkhtmltopdf tool <=0.12.5 + // Plus for old WebKit, typeof returns "function" for HTML collections + // (e.g., `typeof document.getElementsByTagName("div") === "function"`). (gh-4756) + return typeof obj === "function" && typeof obj.nodeType !== "number" && + typeof obj.item !== "function"; + }; + + +var isWindow = function isWindow( obj ) { + return obj != null && obj === obj.window; + }; + + +var document = window.document; + + + + var preservedScriptAttributes = { + type: true, + src: true, + nonce: true, + noModule: true + }; + + function DOMEval( code, node, doc ) { + doc = doc || document; + + var i, val, + script = doc.createElement( "script" ); + + script.text = code; + if ( node ) { + for ( i in preservedScriptAttributes ) { + + // Support: Firefox 64+, Edge 18+ + // Some browsers don't support the "nonce" property on scripts. + // On the other hand, just using `getAttribute` is not enough as + // the `nonce` attribute is reset to an empty string whenever it + // becomes browsing-context connected. + // See https://github.com/whatwg/html/issues/2369 + // See https://html.spec.whatwg.org/#nonce-attributes + // The `node.getAttribute` check was added for the sake of + // `jQuery.globalEval` so that it can fake a nonce-containing node + // via an object. + val = node[ i ] || node.getAttribute && node.getAttribute( i ); + if ( val ) { + script.setAttribute( i, val ); + } + } + } + doc.head.appendChild( script ).parentNode.removeChild( script ); + } + + +function toType( obj ) { + if ( obj == null ) { + return obj + ""; + } + + // Support: Android <=2.3 only (functionish RegExp) + return typeof obj === "object" || typeof obj === "function" ? + class2type[ toString.call( obj ) ] || "object" : + typeof obj; +} +/* global Symbol */ +// Defining this global in .eslintrc.json would create a danger of using the global +// unguarded in another place, it seems safer to define global only for this module + + + +var + version = "3.6.0", + + // Define a local copy of jQuery + jQuery = function( selector, context ) { + + // The jQuery object is actually just the init constructor 'enhanced' + // Need init if jQuery is called (just allow error to be thrown if not included) + return new jQuery.fn.init( selector, context ); + }; + +jQuery.fn = jQuery.prototype = { + + // The current version of jQuery being used + jquery: version, + + constructor: jQuery, + + // The default length of a jQuery object is 0 + length: 0, + + toArray: function() { + return slice.call( this ); + }, + + // Get the Nth element in the matched element set OR + // Get the whole matched element set as a clean array + get: function( num ) { + + // Return all the elements in a clean array + if ( num == null ) { + return slice.call( this ); + } + + // Return just the one element from the set + return num < 0 ? this[ num + this.length ] : this[ num ]; + }, + + // Take an array of elements and push it onto the stack + // (returning the new matched element set) + pushStack: function( elems ) { + + // Build a new jQuery matched element set + var ret = jQuery.merge( this.constructor(), elems ); + + // Add the old object onto the stack (as a reference) + ret.prevObject = this; + + // Return the newly-formed element set + return ret; + }, + + // Execute a callback for every element in the matched set. + each: function( callback ) { + return jQuery.each( this, callback ); + }, + + map: function( callback ) { + return this.pushStack( jQuery.map( this, function( elem, i ) { + return callback.call( elem, i, elem ); + } ) ); + }, + + slice: function() { + return this.pushStack( slice.apply( this, arguments ) ); + }, + + first: function() { + return this.eq( 0 ); + }, + + last: function() { + return this.eq( -1 ); + }, + + even: function() { + return this.pushStack( jQuery.grep( this, function( _elem, i ) { + return ( i + 1 ) % 2; + } ) ); + }, + + odd: function() { + return this.pushStack( jQuery.grep( this, function( _elem, i ) { + return i % 2; + } ) ); + }, + + eq: function( i ) { + var len = this.length, + j = +i + ( i < 0 ? len : 0 ); + return this.pushStack( j >= 0 && j < len ? [ this[ j ] ] : [] ); + }, + + end: function() { + return this.prevObject || this.constructor(); + }, + + // For internal use only. + // Behaves like an Array's method, not like a jQuery method. + push: push, + sort: arr.sort, + splice: arr.splice +}; + +jQuery.extend = jQuery.fn.extend = function() { + var options, name, src, copy, copyIsArray, clone, + target = arguments[ 0 ] || {}, + i = 1, + length = arguments.length, + deep = false; + + // Handle a deep copy situation + if ( typeof target === "boolean" ) { + deep = target; + + // Skip the boolean and the target + target = arguments[ i ] || {}; + i++; + } + + // Handle case when target is a string or something (possible in deep copy) + if ( typeof target !== "object" && !isFunction( target ) ) { + target = {}; + } + + // Extend jQuery itself if only one argument is passed + if ( i === length ) { + target = this; + i--; + } + + for ( ; i < length; i++ ) { + + // Only deal with non-null/undefined values + if ( ( options = arguments[ i ] ) != null ) { + + // Extend the base object + for ( name in options ) { + copy = options[ name ]; + + // Prevent Object.prototype pollution + // Prevent never-ending loop + if ( name === "__proto__" || target === copy ) { + continue; + } + + // Recurse if we're merging plain objects or arrays + if ( deep && copy && ( jQuery.isPlainObject( copy ) || + ( copyIsArray = Array.isArray( copy ) ) ) ) { + src = target[ name ]; + + // Ensure proper type for the source value + if ( copyIsArray && !Array.isArray( src ) ) { + clone = []; + } else if ( !copyIsArray && !jQuery.isPlainObject( src ) ) { + clone = {}; + } else { + clone = src; + } + copyIsArray = false; + + // Never move original objects, clone them + target[ name ] = jQuery.extend( deep, clone, copy ); + + // Don't bring in undefined values + } else if ( copy !== undefined ) { + target[ name ] = copy; + } + } + } + } + + // Return the modified object + return target; +}; + +jQuery.extend( { + + // Unique for each copy of jQuery on the page + expando: "jQuery" + ( version + Math.random() ).replace( /\D/g, "" ), + + // Assume jQuery is ready without the ready module + isReady: true, + + error: function( msg ) { + throw new Error( msg ); + }, + + noop: function() {}, + + isPlainObject: function( obj ) { + var proto, Ctor; + + // Detect obvious negatives + // Use toString instead of jQuery.type to catch host objects + if ( !obj || toString.call( obj ) !== "[object Object]" ) { + return false; + } + + proto = getProto( obj ); + + // Objects with no prototype (e.g., `Object.create( null )`) are plain + if ( !proto ) { + return true; + } + + // Objects with prototype are plain iff they were constructed by a global Object function + Ctor = hasOwn.call( proto, "constructor" ) && proto.constructor; + return typeof Ctor === "function" && fnToString.call( Ctor ) === ObjectFunctionString; + }, + + isEmptyObject: function( obj ) { + var name; + + for ( name in obj ) { + return false; + } + return true; + }, + + // Evaluates a script in a provided context; falls back to the global one + // if not specified. + globalEval: function( code, options, doc ) { + DOMEval( code, { nonce: options && options.nonce }, doc ); + }, + + each: function( obj, callback ) { + var length, i = 0; + + if ( isArrayLike( obj ) ) { + length = obj.length; + for ( ; i < length; i++ ) { + if ( callback.call( obj[ i ], i, obj[ i ] ) === false ) { + break; + } + } + } else { + for ( i in obj ) { + if ( callback.call( obj[ i ], i, obj[ i ] ) === false ) { + break; + } + } + } + + return obj; + }, + + // results is for internal usage only + makeArray: function( arr, results ) { + var ret = results || []; + + if ( arr != null ) { + if ( isArrayLike( Object( arr ) ) ) { + jQuery.merge( ret, + typeof arr === "string" ? + [ arr ] : arr + ); + } else { + push.call( ret, arr ); + } + } + + return ret; + }, + + inArray: function( elem, arr, i ) { + return arr == null ? -1 : indexOf.call( arr, elem, i ); + }, + + // Support: Android <=4.0 only, PhantomJS 1 only + // push.apply(_, arraylike) throws on ancient WebKit + merge: function( first, second ) { + var len = +second.length, + j = 0, + i = first.length; + + for ( ; j < len; j++ ) { + first[ i++ ] = second[ j ]; + } + + first.length = i; + + return first; + }, + + grep: function( elems, callback, invert ) { + var callbackInverse, + matches = [], + i = 0, + length = elems.length, + callbackExpect = !invert; + + // Go through the array, only saving the items + // that pass the validator function + for ( ; i < length; i++ ) { + callbackInverse = !callback( elems[ i ], i ); + if ( callbackInverse !== callbackExpect ) { + matches.push( elems[ i ] ); + } + } + + return matches; + }, + + // arg is for internal usage only + map: function( elems, callback, arg ) { + var length, value, + i = 0, + ret = []; + + // Go through the array, translating each of the items to their new values + if ( isArrayLike( elems ) ) { + length = elems.length; + for ( ; i < length; i++ ) { + value = callback( elems[ i ], i, arg ); + + if ( value != null ) { + ret.push( value ); + } + } + + // Go through every key on the object, + } else { + for ( i in elems ) { + value = callback( elems[ i ], i, arg ); + + if ( value != null ) { + ret.push( value ); + } + } + } + + // Flatten any nested arrays + return flat( ret ); + }, + + // A global GUID counter for objects + guid: 1, + + // jQuery.support is not used in Core but other projects attach their + // properties to it so it needs to exist. + support: support +} ); + +if ( typeof Symbol === "function" ) { + jQuery.fn[ Symbol.iterator ] = arr[ Symbol.iterator ]; +} + +// Populate the class2type map +jQuery.each( "Boolean Number String Function Array Date RegExp Object Error Symbol".split( " " ), + function( _i, name ) { + class2type[ "[object " + name + "]" ] = name.toLowerCase(); + } ); + +function isArrayLike( obj ) { + + // Support: real iOS 8.2 only (not reproducible in simulator) + // `in` check used to prevent JIT error (gh-2145) + // hasOwn isn't used here due to false negatives + // regarding Nodelist length in IE + var length = !!obj && "length" in obj && obj.length, + type = toType( obj ); + + if ( isFunction( obj ) || isWindow( obj ) ) { + return false; + } + + return type === "array" || length === 0 || + typeof length === "number" && length > 0 && ( length - 1 ) in obj; +} +var Sizzle = +/*! + * Sizzle CSS Selector Engine v2.3.6 + * https://sizzlejs.com/ + * + * Copyright JS Foundation and other contributors + * Released under the MIT license + * https://js.foundation/ + * + * Date: 2021-02-16 + */ +( function( window ) { +var i, + support, + Expr, + getText, + isXML, + tokenize, + compile, + select, + outermostContext, + sortInput, + hasDuplicate, + + // Local document vars + setDocument, + document, + docElem, + documentIsHTML, + rbuggyQSA, + rbuggyMatches, + matches, + contains, + + // Instance-specific data + expando = "sizzle" + 1 * new Date(), + preferredDoc = window.document, + dirruns = 0, + done = 0, + classCache = createCache(), + tokenCache = createCache(), + compilerCache = createCache(), + nonnativeSelectorCache = createCache(), + sortOrder = function( a, b ) { + if ( a === b ) { + hasDuplicate = true; + } + return 0; + }, + + // Instance methods + hasOwn = ( {} ).hasOwnProperty, + arr = [], + pop = arr.pop, + pushNative = arr.push, + push = arr.push, + slice = arr.slice, + + // Use a stripped-down indexOf as it's faster than native + // https://jsperf.com/thor-indexof-vs-for/5 + indexOf = function( list, elem ) { + var i = 0, + len = list.length; + for ( ; i < len; i++ ) { + if ( list[ i ] === elem ) { + return i; + } + } + return -1; + }, + + booleans = "checked|selected|async|autofocus|autoplay|controls|defer|disabled|hidden|" + + "ismap|loop|multiple|open|readonly|required|scoped", + + // Regular expressions + + // http://www.w3.org/TR/css3-selectors/#whitespace + whitespace = "[\\x20\\t\\r\\n\\f]", + + // https://www.w3.org/TR/css-syntax-3/#ident-token-diagram + identifier = "(?:\\\\[\\da-fA-F]{1,6}" + whitespace + + "?|\\\\[^\\r\\n\\f]|[\\w-]|[^\0-\\x7f])+", + + // Attribute selectors: http://www.w3.org/TR/selectors/#attribute-selectors + attributes = "\\[" + whitespace + "*(" + identifier + ")(?:" + whitespace + + + // Operator (capture 2) + "*([*^$|!~]?=)" + whitespace + + + // "Attribute values must be CSS identifiers [capture 5] + // or strings [capture 3 or capture 4]" + "*(?:'((?:\\\\.|[^\\\\'])*)'|\"((?:\\\\.|[^\\\\\"])*)\"|(" + identifier + "))|)" + + whitespace + "*\\]", + + pseudos = ":(" + identifier + ")(?:\\((" + + + // To reduce the number of selectors needing tokenize in the preFilter, prefer arguments: + // 1. quoted (capture 3; capture 4 or capture 5) + "('((?:\\\\.|[^\\\\'])*)'|\"((?:\\\\.|[^\\\\\"])*)\")|" + + + // 2. simple (capture 6) + "((?:\\\\.|[^\\\\()[\\]]|" + attributes + ")*)|" + + + // 3. anything else (capture 2) + ".*" + + ")\\)|)", + + // Leading and non-escaped trailing whitespace, capturing some non-whitespace characters preceding the latter + rwhitespace = new RegExp( whitespace + "+", "g" ), + rtrim = new RegExp( "^" + whitespace + "+|((?:^|[^\\\\])(?:\\\\.)*)" + + whitespace + "+$", "g" ), + + rcomma = new RegExp( "^" + whitespace + "*," + whitespace + "*" ), + rcombinators = new RegExp( "^" + whitespace + "*([>+~]|" + whitespace + ")" + whitespace + + "*" ), + rdescend = new RegExp( whitespace + "|>" ), + + rpseudo = new RegExp( pseudos ), + ridentifier = new RegExp( "^" + identifier + "$" ), + + matchExpr = { + "ID": new RegExp( "^#(" + identifier + ")" ), + "CLASS": new RegExp( "^\\.(" + identifier + ")" ), + "TAG": new RegExp( "^(" + identifier + "|[*])" ), + "ATTR": new RegExp( "^" + attributes ), + "PSEUDO": new RegExp( "^" + pseudos ), + "CHILD": new RegExp( "^:(only|first|last|nth|nth-last)-(child|of-type)(?:\\(" + + whitespace + "*(even|odd|(([+-]|)(\\d*)n|)" + whitespace + "*(?:([+-]|)" + + whitespace + "*(\\d+)|))" + whitespace + "*\\)|)", "i" ), + "bool": new RegExp( "^(?:" + booleans + ")$", "i" ), + + // For use in libraries implementing .is() + // We use this for POS matching in `select` + "needsContext": new RegExp( "^" + whitespace + + "*[>+~]|:(even|odd|eq|gt|lt|nth|first|last)(?:\\(" + whitespace + + "*((?:-\\d)?\\d*)" + whitespace + "*\\)|)(?=[^-]|$)", "i" ) + }, + + rhtml = /HTML$/i, + rinputs = /^(?:input|select|textarea|button)$/i, + rheader = /^h\d$/i, + + rnative = /^[^{]+\{\s*\[native \w/, + + // Easily-parseable/retrievable ID or TAG or CLASS selectors + rquickExpr = /^(?:#([\w-]+)|(\w+)|\.([\w-]+))$/, + + rsibling = /[+~]/, + + // CSS escapes + // http://www.w3.org/TR/CSS21/syndata.html#escaped-characters + runescape = new RegExp( "\\\\[\\da-fA-F]{1,6}" + whitespace + "?|\\\\([^\\r\\n\\f])", "g" ), + funescape = function( escape, nonHex ) { + var high = "0x" + escape.slice( 1 ) - 0x10000; + + return nonHex ? + + // Strip the backslash prefix from a non-hex escape sequence + nonHex : + + // Replace a hexadecimal escape sequence with the encoded Unicode code point + // Support: IE <=11+ + // For values outside the Basic Multilingual Plane (BMP), manually construct a + // surrogate pair + high < 0 ? + String.fromCharCode( high + 0x10000 ) : + String.fromCharCode( high >> 10 | 0xD800, high & 0x3FF | 0xDC00 ); + }, + + // CSS string/identifier serialization + // https://drafts.csswg.org/cssom/#common-serializing-idioms + rcssescape = /([\0-\x1f\x7f]|^-?\d)|^-$|[^\0-\x1f\x7f-\uFFFF\w-]/g, + fcssescape = function( ch, asCodePoint ) { + if ( asCodePoint ) { + + // U+0000 NULL becomes U+FFFD REPLACEMENT CHARACTER + if ( ch === "\0" ) { + return "\uFFFD"; + } + + // Control characters and (dependent upon position) numbers get escaped as code points + return ch.slice( 0, -1 ) + "\\" + + ch.charCodeAt( ch.length - 1 ).toString( 16 ) + " "; + } + + // Other potentially-special ASCII characters get backslash-escaped + return "\\" + ch; + }, + + // Used for iframes + // See setDocument() + // Removing the function wrapper causes a "Permission Denied" + // error in IE + unloadHandler = function() { + setDocument(); + }, + + inDisabledFieldset = addCombinator( + function( elem ) { + return elem.disabled === true && elem.nodeName.toLowerCase() === "fieldset"; + }, + { dir: "parentNode", next: "legend" } + ); + +// Optimize for push.apply( _, NodeList ) +try { + push.apply( + ( arr = slice.call( preferredDoc.childNodes ) ), + preferredDoc.childNodes + ); + + // Support: Android<4.0 + // Detect silently failing push.apply + // eslint-disable-next-line no-unused-expressions + arr[ preferredDoc.childNodes.length ].nodeType; +} catch ( e ) { + push = { apply: arr.length ? + + // Leverage slice if possible + function( target, els ) { + pushNative.apply( target, slice.call( els ) ); + } : + + // Support: IE<9 + // Otherwise append directly + function( target, els ) { + var j = target.length, + i = 0; + + // Can't trust NodeList.length + while ( ( target[ j++ ] = els[ i++ ] ) ) {} + target.length = j - 1; + } + }; +} + +function Sizzle( selector, context, results, seed ) { + var m, i, elem, nid, match, groups, newSelector, + newContext = context && context.ownerDocument, + + // nodeType defaults to 9, since context defaults to document + nodeType = context ? context.nodeType : 9; + + results = results || []; + + // Return early from calls with invalid selector or context + if ( typeof selector !== "string" || !selector || + nodeType !== 1 && nodeType !== 9 && nodeType !== 11 ) { + + return results; + } + + // Try to shortcut find operations (as opposed to filters) in HTML documents + if ( !seed ) { + setDocument( context ); + context = context || document; + + if ( documentIsHTML ) { + + // If the selector is sufficiently simple, try using a "get*By*" DOM method + // (excepting DocumentFragment context, where the methods don't exist) + if ( nodeType !== 11 && ( match = rquickExpr.exec( selector ) ) ) { + + // ID selector + if ( ( m = match[ 1 ] ) ) { + + // Document context + if ( nodeType === 9 ) { + if ( ( elem = context.getElementById( m ) ) ) { + + // Support: IE, Opera, Webkit + // TODO: identify versions + // getElementById can match elements by name instead of ID + if ( elem.id === m ) { + results.push( elem ); + return results; + } + } else { + return results; + } + + // Element context + } else { + + // Support: IE, Opera, Webkit + // TODO: identify versions + // getElementById can match elements by name instead of ID + if ( newContext && ( elem = newContext.getElementById( m ) ) && + contains( context, elem ) && + elem.id === m ) { + + results.push( elem ); + return results; + } + } + + // Type selector + } else if ( match[ 2 ] ) { + push.apply( results, context.getElementsByTagName( selector ) ); + return results; + + // Class selector + } else if ( ( m = match[ 3 ] ) && support.getElementsByClassName && + context.getElementsByClassName ) { + + push.apply( results, context.getElementsByClassName( m ) ); + return results; + } + } + + // Take advantage of querySelectorAll + if ( support.qsa && + !nonnativeSelectorCache[ selector + " " ] && + ( !rbuggyQSA || !rbuggyQSA.test( selector ) ) && + + // Support: IE 8 only + // Exclude object elements + ( nodeType !== 1 || context.nodeName.toLowerCase() !== "object" ) ) { + + newSelector = selector; + newContext = context; + + // qSA considers elements outside a scoping root when evaluating child or + // descendant combinators, which is not what we want. + // In such cases, we work around the behavior by prefixing every selector in the + // list with an ID selector referencing the scope context. + // The technique has to be used as well when a leading combinator is used + // as such selectors are not recognized by querySelectorAll. + // Thanks to Andrew Dupont for this technique. + if ( nodeType === 1 && + ( rdescend.test( selector ) || rcombinators.test( selector ) ) ) { + + // Expand context for sibling selectors + newContext = rsibling.test( selector ) && testContext( context.parentNode ) || + context; + + // We can use :scope instead of the ID hack if the browser + // supports it & if we're not changing the context. + if ( newContext !== context || !support.scope ) { + + // Capture the context ID, setting it first if necessary + if ( ( nid = context.getAttribute( "id" ) ) ) { + nid = nid.replace( rcssescape, fcssescape ); + } else { + context.setAttribute( "id", ( nid = expando ) ); + } + } + + // Prefix every selector in the list + groups = tokenize( selector ); + i = groups.length; + while ( i-- ) { + groups[ i ] = ( nid ? "#" + nid : ":scope" ) + " " + + toSelector( groups[ i ] ); + } + newSelector = groups.join( "," ); + } + + try { + push.apply( results, + newContext.querySelectorAll( newSelector ) + ); + return results; + } catch ( qsaError ) { + nonnativeSelectorCache( selector, true ); + } finally { + if ( nid === expando ) { + context.removeAttribute( "id" ); + } + } + } + } + } + + // All others + return select( selector.replace( rtrim, "$1" ), context, results, seed ); +} + +/** + * Create key-value caches of limited size + * @returns {function(string, object)} Returns the Object data after storing it on itself with + * property name the (space-suffixed) string and (if the cache is larger than Expr.cacheLength) + * deleting the oldest entry + */ +function createCache() { + var keys = []; + + function cache( key, value ) { + + // Use (key + " ") to avoid collision with native prototype properties (see Issue #157) + if ( keys.push( key + " " ) > Expr.cacheLength ) { + + // Only keep the most recent entries + delete cache[ keys.shift() ]; + } + return ( cache[ key + " " ] = value ); + } + return cache; +} + +/** + * Mark a function for special use by Sizzle + * @param {Function} fn The function to mark + */ +function markFunction( fn ) { + fn[ expando ] = true; + return fn; +} + +/** + * Support testing using an element + * @param {Function} fn Passed the created element and returns a boolean result + */ +function assert( fn ) { + var el = document.createElement( "fieldset" ); + + try { + return !!fn( el ); + } catch ( e ) { + return false; + } finally { + + // Remove from its parent by default + if ( el.parentNode ) { + el.parentNode.removeChild( el ); + } + + // release memory in IE + el = null; + } +} + +/** + * Adds the same handler for all of the specified attrs + * @param {String} attrs Pipe-separated list of attributes + * @param {Function} handler The method that will be applied + */ +function addHandle( attrs, handler ) { + var arr = attrs.split( "|" ), + i = arr.length; + + while ( i-- ) { + Expr.attrHandle[ arr[ i ] ] = handler; + } +} + +/** + * Checks document order of two siblings + * @param {Element} a + * @param {Element} b + * @returns {Number} Returns less than 0 if a precedes b, greater than 0 if a follows b + */ +function siblingCheck( a, b ) { + var cur = b && a, + diff = cur && a.nodeType === 1 && b.nodeType === 1 && + a.sourceIndex - b.sourceIndex; + + // Use IE sourceIndex if available on both nodes + if ( diff ) { + return diff; + } + + // Check if b follows a + if ( cur ) { + while ( ( cur = cur.nextSibling ) ) { + if ( cur === b ) { + return -1; + } + } + } + + return a ? 1 : -1; +} + +/** + * Returns a function to use in pseudos for input types + * @param {String} type + */ +function createInputPseudo( type ) { + return function( elem ) { + var name = elem.nodeName.toLowerCase(); + return name === "input" && elem.type === type; + }; +} + +/** + * Returns a function to use in pseudos for buttons + * @param {String} type + */ +function createButtonPseudo( type ) { + return function( elem ) { + var name = elem.nodeName.toLowerCase(); + return ( name === "input" || name === "button" ) && elem.type === type; + }; +} + +/** + * Returns a function to use in pseudos for :enabled/:disabled + * @param {Boolean} disabled true for :disabled; false for :enabled + */ +function createDisabledPseudo( disabled ) { + + // Known :disabled false positives: fieldset[disabled] > legend:nth-of-type(n+2) :can-disable + return function( elem ) { + + // Only certain elements can match :enabled or :disabled + // https://html.spec.whatwg.org/multipage/scripting.html#selector-enabled + // https://html.spec.whatwg.org/multipage/scripting.html#selector-disabled + if ( "form" in elem ) { + + // Check for inherited disabledness on relevant non-disabled elements: + // * listed form-associated elements in a disabled fieldset + // https://html.spec.whatwg.org/multipage/forms.html#category-listed + // https://html.spec.whatwg.org/multipage/forms.html#concept-fe-disabled + // * option elements in a disabled optgroup + // https://html.spec.whatwg.org/multipage/forms.html#concept-option-disabled + // All such elements have a "form" property. + if ( elem.parentNode && elem.disabled === false ) { + + // Option elements defer to a parent optgroup if present + if ( "label" in elem ) { + if ( "label" in elem.parentNode ) { + return elem.parentNode.disabled === disabled; + } else { + return elem.disabled === disabled; + } + } + + // Support: IE 6 - 11 + // Use the isDisabled shortcut property to check for disabled fieldset ancestors + return elem.isDisabled === disabled || + + // Where there is no isDisabled, check manually + /* jshint -W018 */ + elem.isDisabled !== !disabled && + inDisabledFieldset( elem ) === disabled; + } + + return elem.disabled === disabled; + + // Try to winnow out elements that can't be disabled before trusting the disabled property. + // Some victims get caught in our net (label, legend, menu, track), but it shouldn't + // even exist on them, let alone have a boolean value. + } else if ( "label" in elem ) { + return elem.disabled === disabled; + } + + // Remaining elements are neither :enabled nor :disabled + return false; + }; +} + +/** + * Returns a function to use in pseudos for positionals + * @param {Function} fn + */ +function createPositionalPseudo( fn ) { + return markFunction( function( argument ) { + argument = +argument; + return markFunction( function( seed, matches ) { + var j, + matchIndexes = fn( [], seed.length, argument ), + i = matchIndexes.length; + + // Match elements found at the specified indexes + while ( i-- ) { + if ( seed[ ( j = matchIndexes[ i ] ) ] ) { + seed[ j ] = !( matches[ j ] = seed[ j ] ); + } + } + } ); + } ); +} + +/** + * Checks a node for validity as a Sizzle context + * @param {Element|Object=} context + * @returns {Element|Object|Boolean} The input node if acceptable, otherwise a falsy value + */ +function testContext( context ) { + return context && typeof context.getElementsByTagName !== "undefined" && context; +} + +// Expose support vars for convenience +support = Sizzle.support = {}; + +/** + * Detects XML nodes + * @param {Element|Object} elem An element or a document + * @returns {Boolean} True iff elem is a non-HTML XML node + */ +isXML = Sizzle.isXML = function( elem ) { + var namespace = elem && elem.namespaceURI, + docElem = elem && ( elem.ownerDocument || elem ).documentElement; + + // Support: IE <=8 + // Assume HTML when documentElement doesn't yet exist, such as inside loading iframes + // https://bugs.jquery.com/ticket/4833 + return !rhtml.test( namespace || docElem && docElem.nodeName || "HTML" ); +}; + +/** + * Sets document-related variables once based on the current document + * @param {Element|Object} [doc] An element or document object to use to set the document + * @returns {Object} Returns the current document + */ +setDocument = Sizzle.setDocument = function( node ) { + var hasCompare, subWindow, + doc = node ? node.ownerDocument || node : preferredDoc; + + // Return early if doc is invalid or already selected + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + if ( doc == document || doc.nodeType !== 9 || !doc.documentElement ) { + return document; + } + + // Update global variables + document = doc; + docElem = document.documentElement; + documentIsHTML = !isXML( document ); + + // Support: IE 9 - 11+, Edge 12 - 18+ + // Accessing iframe documents after unload throws "permission denied" errors (jQuery #13936) + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + if ( preferredDoc != document && + ( subWindow = document.defaultView ) && subWindow.top !== subWindow ) { + + // Support: IE 11, Edge + if ( subWindow.addEventListener ) { + subWindow.addEventListener( "unload", unloadHandler, false ); + + // Support: IE 9 - 10 only + } else if ( subWindow.attachEvent ) { + subWindow.attachEvent( "onunload", unloadHandler ); + } + } + + // Support: IE 8 - 11+, Edge 12 - 18+, Chrome <=16 - 25 only, Firefox <=3.6 - 31 only, + // Safari 4 - 5 only, Opera <=11.6 - 12.x only + // IE/Edge & older browsers don't support the :scope pseudo-class. + // Support: Safari 6.0 only + // Safari 6.0 supports :scope but it's an alias of :root there. + support.scope = assert( function( el ) { + docElem.appendChild( el ).appendChild( document.createElement( "div" ) ); + return typeof el.querySelectorAll !== "undefined" && + !el.querySelectorAll( ":scope fieldset div" ).length; + } ); + + /* Attributes + ---------------------------------------------------------------------- */ + + // Support: IE<8 + // Verify that getAttribute really returns attributes and not properties + // (excepting IE8 booleans) + support.attributes = assert( function( el ) { + el.className = "i"; + return !el.getAttribute( "className" ); + } ); + + /* getElement(s)By* + ---------------------------------------------------------------------- */ + + // Check if getElementsByTagName("*") returns only elements + support.getElementsByTagName = assert( function( el ) { + el.appendChild( document.createComment( "" ) ); + return !el.getElementsByTagName( "*" ).length; + } ); + + // Support: IE<9 + support.getElementsByClassName = rnative.test( document.getElementsByClassName ); + + // Support: IE<10 + // Check if getElementById returns elements by name + // The broken getElementById methods don't pick up programmatically-set names, + // so use a roundabout getElementsByName test + support.getById = assert( function( el ) { + docElem.appendChild( el ).id = expando; + return !document.getElementsByName || !document.getElementsByName( expando ).length; + } ); + + // ID filter and find + if ( support.getById ) { + Expr.filter[ "ID" ] = function( id ) { + var attrId = id.replace( runescape, funescape ); + return function( elem ) { + return elem.getAttribute( "id" ) === attrId; + }; + }; + Expr.find[ "ID" ] = function( id, context ) { + if ( typeof context.getElementById !== "undefined" && documentIsHTML ) { + var elem = context.getElementById( id ); + return elem ? [ elem ] : []; + } + }; + } else { + Expr.filter[ "ID" ] = function( id ) { + var attrId = id.replace( runescape, funescape ); + return function( elem ) { + var node = typeof elem.getAttributeNode !== "undefined" && + elem.getAttributeNode( "id" ); + return node && node.value === attrId; + }; + }; + + // Support: IE 6 - 7 only + // getElementById is not reliable as a find shortcut + Expr.find[ "ID" ] = function( id, context ) { + if ( typeof context.getElementById !== "undefined" && documentIsHTML ) { + var node, i, elems, + elem = context.getElementById( id ); + + if ( elem ) { + + // Verify the id attribute + node = elem.getAttributeNode( "id" ); + if ( node && node.value === id ) { + return [ elem ]; + } + + // Fall back on getElementsByName + elems = context.getElementsByName( id ); + i = 0; + while ( ( elem = elems[ i++ ] ) ) { + node = elem.getAttributeNode( "id" ); + if ( node && node.value === id ) { + return [ elem ]; + } + } + } + + return []; + } + }; + } + + // Tag + Expr.find[ "TAG" ] = support.getElementsByTagName ? + function( tag, context ) { + if ( typeof context.getElementsByTagName !== "undefined" ) { + return context.getElementsByTagName( tag ); + + // DocumentFragment nodes don't have gEBTN + } else if ( support.qsa ) { + return context.querySelectorAll( tag ); + } + } : + + function( tag, context ) { + var elem, + tmp = [], + i = 0, + + // By happy coincidence, a (broken) gEBTN appears on DocumentFragment nodes too + results = context.getElementsByTagName( tag ); + + // Filter out possible comments + if ( tag === "*" ) { + while ( ( elem = results[ i++ ] ) ) { + if ( elem.nodeType === 1 ) { + tmp.push( elem ); + } + } + + return tmp; + } + return results; + }; + + // Class + Expr.find[ "CLASS" ] = support.getElementsByClassName && function( className, context ) { + if ( typeof context.getElementsByClassName !== "undefined" && documentIsHTML ) { + return context.getElementsByClassName( className ); + } + }; + + /* QSA/matchesSelector + ---------------------------------------------------------------------- */ + + // QSA and matchesSelector support + + // matchesSelector(:active) reports false when true (IE9/Opera 11.5) + rbuggyMatches = []; + + // qSa(:focus) reports false when true (Chrome 21) + // We allow this because of a bug in IE8/9 that throws an error + // whenever `document.activeElement` is accessed on an iframe + // So, we allow :focus to pass through QSA all the time to avoid the IE error + // See https://bugs.jquery.com/ticket/13378 + rbuggyQSA = []; + + if ( ( support.qsa = rnative.test( document.querySelectorAll ) ) ) { + + // Build QSA regex + // Regex strategy adopted from Diego Perini + assert( function( el ) { + + var input; + + // Select is set to empty string on purpose + // This is to test IE's treatment of not explicitly + // setting a boolean content attribute, + // since its presence should be enough + // https://bugs.jquery.com/ticket/12359 + docElem.appendChild( el ).innerHTML = "" + + ""; + + // Support: IE8, Opera 11-12.16 + // Nothing should be selected when empty strings follow ^= or $= or *= + // The test attribute must be unknown in Opera but "safe" for WinRT + // https://msdn.microsoft.com/en-us/library/ie/hh465388.aspx#attribute_section + if ( el.querySelectorAll( "[msallowcapture^='']" ).length ) { + rbuggyQSA.push( "[*^$]=" + whitespace + "*(?:''|\"\")" ); + } + + // Support: IE8 + // Boolean attributes and "value" are not treated correctly + if ( !el.querySelectorAll( "[selected]" ).length ) { + rbuggyQSA.push( "\\[" + whitespace + "*(?:value|" + booleans + ")" ); + } + + // Support: Chrome<29, Android<4.4, Safari<7.0+, iOS<7.0+, PhantomJS<1.9.8+ + if ( !el.querySelectorAll( "[id~=" + expando + "-]" ).length ) { + rbuggyQSA.push( "~=" ); + } + + // Support: IE 11+, Edge 15 - 18+ + // IE 11/Edge don't find elements on a `[name='']` query in some cases. + // Adding a temporary attribute to the document before the selection works + // around the issue. + // Interestingly, IE 10 & older don't seem to have the issue. + input = document.createElement( "input" ); + input.setAttribute( "name", "" ); + el.appendChild( input ); + if ( !el.querySelectorAll( "[name='']" ).length ) { + rbuggyQSA.push( "\\[" + whitespace + "*name" + whitespace + "*=" + + whitespace + "*(?:''|\"\")" ); + } + + // Webkit/Opera - :checked should return selected option elements + // http://www.w3.org/TR/2011/REC-css3-selectors-20110929/#checked + // IE8 throws error here and will not see later tests + if ( !el.querySelectorAll( ":checked" ).length ) { + rbuggyQSA.push( ":checked" ); + } + + // Support: Safari 8+, iOS 8+ + // https://bugs.webkit.org/show_bug.cgi?id=136851 + // In-page `selector#id sibling-combinator selector` fails + if ( !el.querySelectorAll( "a#" + expando + "+*" ).length ) { + rbuggyQSA.push( ".#.+[+~]" ); + } + + // Support: Firefox <=3.6 - 5 only + // Old Firefox doesn't throw on a badly-escaped identifier. + el.querySelectorAll( "\\\f" ); + rbuggyQSA.push( "[\\r\\n\\f]" ); + } ); + + assert( function( el ) { + el.innerHTML = "" + + ""; + + // Support: Windows 8 Native Apps + // The type and name attributes are restricted during .innerHTML assignment + var input = document.createElement( "input" ); + input.setAttribute( "type", "hidden" ); + el.appendChild( input ).setAttribute( "name", "D" ); + + // Support: IE8 + // Enforce case-sensitivity of name attribute + if ( el.querySelectorAll( "[name=d]" ).length ) { + rbuggyQSA.push( "name" + whitespace + "*[*^$|!~]?=" ); + } + + // FF 3.5 - :enabled/:disabled and hidden elements (hidden elements are still enabled) + // IE8 throws error here and will not see later tests + if ( el.querySelectorAll( ":enabled" ).length !== 2 ) { + rbuggyQSA.push( ":enabled", ":disabled" ); + } + + // Support: IE9-11+ + // IE's :disabled selector does not pick up the children of disabled fieldsets + docElem.appendChild( el ).disabled = true; + if ( el.querySelectorAll( ":disabled" ).length !== 2 ) { + rbuggyQSA.push( ":enabled", ":disabled" ); + } + + // Support: Opera 10 - 11 only + // Opera 10-11 does not throw on post-comma invalid pseudos + el.querySelectorAll( "*,:x" ); + rbuggyQSA.push( ",.*:" ); + } ); + } + + if ( ( support.matchesSelector = rnative.test( ( matches = docElem.matches || + docElem.webkitMatchesSelector || + docElem.mozMatchesSelector || + docElem.oMatchesSelector || + docElem.msMatchesSelector ) ) ) ) { + + assert( function( el ) { + + // Check to see if it's possible to do matchesSelector + // on a disconnected node (IE 9) + support.disconnectedMatch = matches.call( el, "*" ); + + // This should fail with an exception + // Gecko does not error, returns false instead + matches.call( el, "[s!='']:x" ); + rbuggyMatches.push( "!=", pseudos ); + } ); + } + + rbuggyQSA = rbuggyQSA.length && new RegExp( rbuggyQSA.join( "|" ) ); + rbuggyMatches = rbuggyMatches.length && new RegExp( rbuggyMatches.join( "|" ) ); + + /* Contains + ---------------------------------------------------------------------- */ + hasCompare = rnative.test( docElem.compareDocumentPosition ); + + // Element contains another + // Purposefully self-exclusive + // As in, an element does not contain itself + contains = hasCompare || rnative.test( docElem.contains ) ? + function( a, b ) { + var adown = a.nodeType === 9 ? a.documentElement : a, + bup = b && b.parentNode; + return a === bup || !!( bup && bup.nodeType === 1 && ( + adown.contains ? + adown.contains( bup ) : + a.compareDocumentPosition && a.compareDocumentPosition( bup ) & 16 + ) ); + } : + function( a, b ) { + if ( b ) { + while ( ( b = b.parentNode ) ) { + if ( b === a ) { + return true; + } + } + } + return false; + }; + + /* Sorting + ---------------------------------------------------------------------- */ + + // Document order sorting + sortOrder = hasCompare ? + function( a, b ) { + + // Flag for duplicate removal + if ( a === b ) { + hasDuplicate = true; + return 0; + } + + // Sort on method existence if only one input has compareDocumentPosition + var compare = !a.compareDocumentPosition - !b.compareDocumentPosition; + if ( compare ) { + return compare; + } + + // Calculate position if both inputs belong to the same document + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + compare = ( a.ownerDocument || a ) == ( b.ownerDocument || b ) ? + a.compareDocumentPosition( b ) : + + // Otherwise we know they are disconnected + 1; + + // Disconnected nodes + if ( compare & 1 || + ( !support.sortDetached && b.compareDocumentPosition( a ) === compare ) ) { + + // Choose the first element that is related to our preferred document + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + if ( a == document || a.ownerDocument == preferredDoc && + contains( preferredDoc, a ) ) { + return -1; + } + + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + if ( b == document || b.ownerDocument == preferredDoc && + contains( preferredDoc, b ) ) { + return 1; + } + + // Maintain original order + return sortInput ? + ( indexOf( sortInput, a ) - indexOf( sortInput, b ) ) : + 0; + } + + return compare & 4 ? -1 : 1; + } : + function( a, b ) { + + // Exit early if the nodes are identical + if ( a === b ) { + hasDuplicate = true; + return 0; + } + + var cur, + i = 0, + aup = a.parentNode, + bup = b.parentNode, + ap = [ a ], + bp = [ b ]; + + // Parentless nodes are either documents or disconnected + if ( !aup || !bup ) { + + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + /* eslint-disable eqeqeq */ + return a == document ? -1 : + b == document ? 1 : + /* eslint-enable eqeqeq */ + aup ? -1 : + bup ? 1 : + sortInput ? + ( indexOf( sortInput, a ) - indexOf( sortInput, b ) ) : + 0; + + // If the nodes are siblings, we can do a quick check + } else if ( aup === bup ) { + return siblingCheck( a, b ); + } + + // Otherwise we need full lists of their ancestors for comparison + cur = a; + while ( ( cur = cur.parentNode ) ) { + ap.unshift( cur ); + } + cur = b; + while ( ( cur = cur.parentNode ) ) { + bp.unshift( cur ); + } + + // Walk down the tree looking for a discrepancy + while ( ap[ i ] === bp[ i ] ) { + i++; + } + + return i ? + + // Do a sibling check if the nodes have a common ancestor + siblingCheck( ap[ i ], bp[ i ] ) : + + // Otherwise nodes in our document sort first + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + /* eslint-disable eqeqeq */ + ap[ i ] == preferredDoc ? -1 : + bp[ i ] == preferredDoc ? 1 : + /* eslint-enable eqeqeq */ + 0; + }; + + return document; +}; + +Sizzle.matches = function( expr, elements ) { + return Sizzle( expr, null, null, elements ); +}; + +Sizzle.matchesSelector = function( elem, expr ) { + setDocument( elem ); + + if ( support.matchesSelector && documentIsHTML && + !nonnativeSelectorCache[ expr + " " ] && + ( !rbuggyMatches || !rbuggyMatches.test( expr ) ) && + ( !rbuggyQSA || !rbuggyQSA.test( expr ) ) ) { + + try { + var ret = matches.call( elem, expr ); + + // IE 9's matchesSelector returns false on disconnected nodes + if ( ret || support.disconnectedMatch || + + // As well, disconnected nodes are said to be in a document + // fragment in IE 9 + elem.document && elem.document.nodeType !== 11 ) { + return ret; + } + } catch ( e ) { + nonnativeSelectorCache( expr, true ); + } + } + + return Sizzle( expr, document, null, [ elem ] ).length > 0; +}; + +Sizzle.contains = function( context, elem ) { + + // Set document vars if needed + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + if ( ( context.ownerDocument || context ) != document ) { + setDocument( context ); + } + return contains( context, elem ); +}; + +Sizzle.attr = function( elem, name ) { + + // Set document vars if needed + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + if ( ( elem.ownerDocument || elem ) != document ) { + setDocument( elem ); + } + + var fn = Expr.attrHandle[ name.toLowerCase() ], + + // Don't get fooled by Object.prototype properties (jQuery #13807) + val = fn && hasOwn.call( Expr.attrHandle, name.toLowerCase() ) ? + fn( elem, name, !documentIsHTML ) : + undefined; + + return val !== undefined ? + val : + support.attributes || !documentIsHTML ? + elem.getAttribute( name ) : + ( val = elem.getAttributeNode( name ) ) && val.specified ? + val.value : + null; +}; + +Sizzle.escape = function( sel ) { + return ( sel + "" ).replace( rcssescape, fcssescape ); +}; + +Sizzle.error = function( msg ) { + throw new Error( "Syntax error, unrecognized expression: " + msg ); +}; + +/** + * Document sorting and removing duplicates + * @param {ArrayLike} results + */ +Sizzle.uniqueSort = function( results ) { + var elem, + duplicates = [], + j = 0, + i = 0; + + // Unless we *know* we can detect duplicates, assume their presence + hasDuplicate = !support.detectDuplicates; + sortInput = !support.sortStable && results.slice( 0 ); + results.sort( sortOrder ); + + if ( hasDuplicate ) { + while ( ( elem = results[ i++ ] ) ) { + if ( elem === results[ i ] ) { + j = duplicates.push( i ); + } + } + while ( j-- ) { + results.splice( duplicates[ j ], 1 ); + } + } + + // Clear input after sorting to release objects + // See https://github.com/jquery/sizzle/pull/225 + sortInput = null; + + return results; +}; + +/** + * Utility function for retrieving the text value of an array of DOM nodes + * @param {Array|Element} elem + */ +getText = Sizzle.getText = function( elem ) { + var node, + ret = "", + i = 0, + nodeType = elem.nodeType; + + if ( !nodeType ) { + + // If no nodeType, this is expected to be an array + while ( ( node = elem[ i++ ] ) ) { + + // Do not traverse comment nodes + ret += getText( node ); + } + } else if ( nodeType === 1 || nodeType === 9 || nodeType === 11 ) { + + // Use textContent for elements + // innerText usage removed for consistency of new lines (jQuery #11153) + if ( typeof elem.textContent === "string" ) { + return elem.textContent; + } else { + + // Traverse its children + for ( elem = elem.firstChild; elem; elem = elem.nextSibling ) { + ret += getText( elem ); + } + } + } else if ( nodeType === 3 || nodeType === 4 ) { + return elem.nodeValue; + } + + // Do not include comment or processing instruction nodes + + return ret; +}; + +Expr = Sizzle.selectors = { + + // Can be adjusted by the user + cacheLength: 50, + + createPseudo: markFunction, + + match: matchExpr, + + attrHandle: {}, + + find: {}, + + relative: { + ">": { dir: "parentNode", first: true }, + " ": { dir: "parentNode" }, + "+": { dir: "previousSibling", first: true }, + "~": { dir: "previousSibling" } + }, + + preFilter: { + "ATTR": function( match ) { + match[ 1 ] = match[ 1 ].replace( runescape, funescape ); + + // Move the given value to match[3] whether quoted or unquoted + match[ 3 ] = ( match[ 3 ] || match[ 4 ] || + match[ 5 ] || "" ).replace( runescape, funescape ); + + if ( match[ 2 ] === "~=" ) { + match[ 3 ] = " " + match[ 3 ] + " "; + } + + return match.slice( 0, 4 ); + }, + + "CHILD": function( match ) { + + /* matches from matchExpr["CHILD"] + 1 type (only|nth|...) + 2 what (child|of-type) + 3 argument (even|odd|\d*|\d*n([+-]\d+)?|...) + 4 xn-component of xn+y argument ([+-]?\d*n|) + 5 sign of xn-component + 6 x of xn-component + 7 sign of y-component + 8 y of y-component + */ + match[ 1 ] = match[ 1 ].toLowerCase(); + + if ( match[ 1 ].slice( 0, 3 ) === "nth" ) { + + // nth-* requires argument + if ( !match[ 3 ] ) { + Sizzle.error( match[ 0 ] ); + } + + // numeric x and y parameters for Expr.filter.CHILD + // remember that false/true cast respectively to 0/1 + match[ 4 ] = +( match[ 4 ] ? + match[ 5 ] + ( match[ 6 ] || 1 ) : + 2 * ( match[ 3 ] === "even" || match[ 3 ] === "odd" ) ); + match[ 5 ] = +( ( match[ 7 ] + match[ 8 ] ) || match[ 3 ] === "odd" ); + + // other types prohibit arguments + } else if ( match[ 3 ] ) { + Sizzle.error( match[ 0 ] ); + } + + return match; + }, + + "PSEUDO": function( match ) { + var excess, + unquoted = !match[ 6 ] && match[ 2 ]; + + if ( matchExpr[ "CHILD" ].test( match[ 0 ] ) ) { + return null; + } + + // Accept quoted arguments as-is + if ( match[ 3 ] ) { + match[ 2 ] = match[ 4 ] || match[ 5 ] || ""; + + // Strip excess characters from unquoted arguments + } else if ( unquoted && rpseudo.test( unquoted ) && + + // Get excess from tokenize (recursively) + ( excess = tokenize( unquoted, true ) ) && + + // advance to the next closing parenthesis + ( excess = unquoted.indexOf( ")", unquoted.length - excess ) - unquoted.length ) ) { + + // excess is a negative index + match[ 0 ] = match[ 0 ].slice( 0, excess ); + match[ 2 ] = unquoted.slice( 0, excess ); + } + + // Return only captures needed by the pseudo filter method (type and argument) + return match.slice( 0, 3 ); + } + }, + + filter: { + + "TAG": function( nodeNameSelector ) { + var nodeName = nodeNameSelector.replace( runescape, funescape ).toLowerCase(); + return nodeNameSelector === "*" ? + function() { + return true; + } : + function( elem ) { + return elem.nodeName && elem.nodeName.toLowerCase() === nodeName; + }; + }, + + "CLASS": function( className ) { + var pattern = classCache[ className + " " ]; + + return pattern || + ( pattern = new RegExp( "(^|" + whitespace + + ")" + className + "(" + whitespace + "|$)" ) ) && classCache( + className, function( elem ) { + return pattern.test( + typeof elem.className === "string" && elem.className || + typeof elem.getAttribute !== "undefined" && + elem.getAttribute( "class" ) || + "" + ); + } ); + }, + + "ATTR": function( name, operator, check ) { + return function( elem ) { + var result = Sizzle.attr( elem, name ); + + if ( result == null ) { + return operator === "!="; + } + if ( !operator ) { + return true; + } + + result += ""; + + /* eslint-disable max-len */ + + return operator === "=" ? result === check : + operator === "!=" ? result !== check : + operator === "^=" ? check && result.indexOf( check ) === 0 : + operator === "*=" ? check && result.indexOf( check ) > -1 : + operator === "$=" ? check && result.slice( -check.length ) === check : + operator === "~=" ? ( " " + result.replace( rwhitespace, " " ) + " " ).indexOf( check ) > -1 : + operator === "|=" ? result === check || result.slice( 0, check.length + 1 ) === check + "-" : + false; + /* eslint-enable max-len */ + + }; + }, + + "CHILD": function( type, what, _argument, first, last ) { + var simple = type.slice( 0, 3 ) !== "nth", + forward = type.slice( -4 ) !== "last", + ofType = what === "of-type"; + + return first === 1 && last === 0 ? + + // Shortcut for :nth-*(n) + function( elem ) { + return !!elem.parentNode; + } : + + function( elem, _context, xml ) { + var cache, uniqueCache, outerCache, node, nodeIndex, start, + dir = simple !== forward ? "nextSibling" : "previousSibling", + parent = elem.parentNode, + name = ofType && elem.nodeName.toLowerCase(), + useCache = !xml && !ofType, + diff = false; + + if ( parent ) { + + // :(first|last|only)-(child|of-type) + if ( simple ) { + while ( dir ) { + node = elem; + while ( ( node = node[ dir ] ) ) { + if ( ofType ? + node.nodeName.toLowerCase() === name : + node.nodeType === 1 ) { + + return false; + } + } + + // Reverse direction for :only-* (if we haven't yet done so) + start = dir = type === "only" && !start && "nextSibling"; + } + return true; + } + + start = [ forward ? parent.firstChild : parent.lastChild ]; + + // non-xml :nth-child(...) stores cache data on `parent` + if ( forward && useCache ) { + + // Seek `elem` from a previously-cached index + + // ...in a gzip-friendly way + node = parent; + outerCache = node[ expando ] || ( node[ expando ] = {} ); + + // Support: IE <9 only + // Defend against cloned attroperties (jQuery gh-1709) + uniqueCache = outerCache[ node.uniqueID ] || + ( outerCache[ node.uniqueID ] = {} ); + + cache = uniqueCache[ type ] || []; + nodeIndex = cache[ 0 ] === dirruns && cache[ 1 ]; + diff = nodeIndex && cache[ 2 ]; + node = nodeIndex && parent.childNodes[ nodeIndex ]; + + while ( ( node = ++nodeIndex && node && node[ dir ] || + + // Fallback to seeking `elem` from the start + ( diff = nodeIndex = 0 ) || start.pop() ) ) { + + // When found, cache indexes on `parent` and break + if ( node.nodeType === 1 && ++diff && node === elem ) { + uniqueCache[ type ] = [ dirruns, nodeIndex, diff ]; + break; + } + } + + } else { + + // Use previously-cached element index if available + if ( useCache ) { + + // ...in a gzip-friendly way + node = elem; + outerCache = node[ expando ] || ( node[ expando ] = {} ); + + // Support: IE <9 only + // Defend against cloned attroperties (jQuery gh-1709) + uniqueCache = outerCache[ node.uniqueID ] || + ( outerCache[ node.uniqueID ] = {} ); + + cache = uniqueCache[ type ] || []; + nodeIndex = cache[ 0 ] === dirruns && cache[ 1 ]; + diff = nodeIndex; + } + + // xml :nth-child(...) + // or :nth-last-child(...) or :nth(-last)?-of-type(...) + if ( diff === false ) { + + // Use the same loop as above to seek `elem` from the start + while ( ( node = ++nodeIndex && node && node[ dir ] || + ( diff = nodeIndex = 0 ) || start.pop() ) ) { + + if ( ( ofType ? + node.nodeName.toLowerCase() === name : + node.nodeType === 1 ) && + ++diff ) { + + // Cache the index of each encountered element + if ( useCache ) { + outerCache = node[ expando ] || + ( node[ expando ] = {} ); + + // Support: IE <9 only + // Defend against cloned attroperties (jQuery gh-1709) + uniqueCache = outerCache[ node.uniqueID ] || + ( outerCache[ node.uniqueID ] = {} ); + + uniqueCache[ type ] = [ dirruns, diff ]; + } + + if ( node === elem ) { + break; + } + } + } + } + } + + // Incorporate the offset, then check against cycle size + diff -= last; + return diff === first || ( diff % first === 0 && diff / first >= 0 ); + } + }; + }, + + "PSEUDO": function( pseudo, argument ) { + + // pseudo-class names are case-insensitive + // http://www.w3.org/TR/selectors/#pseudo-classes + // Prioritize by case sensitivity in case custom pseudos are added with uppercase letters + // Remember that setFilters inherits from pseudos + var args, + fn = Expr.pseudos[ pseudo ] || Expr.setFilters[ pseudo.toLowerCase() ] || + Sizzle.error( "unsupported pseudo: " + pseudo ); + + // The user may use createPseudo to indicate that + // arguments are needed to create the filter function + // just as Sizzle does + if ( fn[ expando ] ) { + return fn( argument ); + } + + // But maintain support for old signatures + if ( fn.length > 1 ) { + args = [ pseudo, pseudo, "", argument ]; + return Expr.setFilters.hasOwnProperty( pseudo.toLowerCase() ) ? + markFunction( function( seed, matches ) { + var idx, + matched = fn( seed, argument ), + i = matched.length; + while ( i-- ) { + idx = indexOf( seed, matched[ i ] ); + seed[ idx ] = !( matches[ idx ] = matched[ i ] ); + } + } ) : + function( elem ) { + return fn( elem, 0, args ); + }; + } + + return fn; + } + }, + + pseudos: { + + // Potentially complex pseudos + "not": markFunction( function( selector ) { + + // Trim the selector passed to compile + // to avoid treating leading and trailing + // spaces as combinators + var input = [], + results = [], + matcher = compile( selector.replace( rtrim, "$1" ) ); + + return matcher[ expando ] ? + markFunction( function( seed, matches, _context, xml ) { + var elem, + unmatched = matcher( seed, null, xml, [] ), + i = seed.length; + + // Match elements unmatched by `matcher` + while ( i-- ) { + if ( ( elem = unmatched[ i ] ) ) { + seed[ i ] = !( matches[ i ] = elem ); + } + } + } ) : + function( elem, _context, xml ) { + input[ 0 ] = elem; + matcher( input, null, xml, results ); + + // Don't keep the element (issue #299) + input[ 0 ] = null; + return !results.pop(); + }; + } ), + + "has": markFunction( function( selector ) { + return function( elem ) { + return Sizzle( selector, elem ).length > 0; + }; + } ), + + "contains": markFunction( function( text ) { + text = text.replace( runescape, funescape ); + return function( elem ) { + return ( elem.textContent || getText( elem ) ).indexOf( text ) > -1; + }; + } ), + + // "Whether an element is represented by a :lang() selector + // is based solely on the element's language value + // being equal to the identifier C, + // or beginning with the identifier C immediately followed by "-". + // The matching of C against the element's language value is performed case-insensitively. + // The identifier C does not have to be a valid language name." + // http://www.w3.org/TR/selectors/#lang-pseudo + "lang": markFunction( function( lang ) { + + // lang value must be a valid identifier + if ( !ridentifier.test( lang || "" ) ) { + Sizzle.error( "unsupported lang: " + lang ); + } + lang = lang.replace( runescape, funescape ).toLowerCase(); + return function( elem ) { + var elemLang; + do { + if ( ( elemLang = documentIsHTML ? + elem.lang : + elem.getAttribute( "xml:lang" ) || elem.getAttribute( "lang" ) ) ) { + + elemLang = elemLang.toLowerCase(); + return elemLang === lang || elemLang.indexOf( lang + "-" ) === 0; + } + } while ( ( elem = elem.parentNode ) && elem.nodeType === 1 ); + return false; + }; + } ), + + // Miscellaneous + "target": function( elem ) { + var hash = window.location && window.location.hash; + return hash && hash.slice( 1 ) === elem.id; + }, + + "root": function( elem ) { + return elem === docElem; + }, + + "focus": function( elem ) { + return elem === document.activeElement && + ( !document.hasFocus || document.hasFocus() ) && + !!( elem.type || elem.href || ~elem.tabIndex ); + }, + + // Boolean properties + "enabled": createDisabledPseudo( false ), + "disabled": createDisabledPseudo( true ), + + "checked": function( elem ) { + + // In CSS3, :checked should return both checked and selected elements + // http://www.w3.org/TR/2011/REC-css3-selectors-20110929/#checked + var nodeName = elem.nodeName.toLowerCase(); + return ( nodeName === "input" && !!elem.checked ) || + ( nodeName === "option" && !!elem.selected ); + }, + + "selected": function( elem ) { + + // Accessing this property makes selected-by-default + // options in Safari work properly + if ( elem.parentNode ) { + // eslint-disable-next-line no-unused-expressions + elem.parentNode.selectedIndex; + } + + return elem.selected === true; + }, + + // Contents + "empty": function( elem ) { + + // http://www.w3.org/TR/selectors/#empty-pseudo + // :empty is negated by element (1) or content nodes (text: 3; cdata: 4; entity ref: 5), + // but not by others (comment: 8; processing instruction: 7; etc.) + // nodeType < 6 works because attributes (2) do not appear as children + for ( elem = elem.firstChild; elem; elem = elem.nextSibling ) { + if ( elem.nodeType < 6 ) { + return false; + } + } + return true; + }, + + "parent": function( elem ) { + return !Expr.pseudos[ "empty" ]( elem ); + }, + + // Element/input types + "header": function( elem ) { + return rheader.test( elem.nodeName ); + }, + + "input": function( elem ) { + return rinputs.test( elem.nodeName ); + }, + + "button": function( elem ) { + var name = elem.nodeName.toLowerCase(); + return name === "input" && elem.type === "button" || name === "button"; + }, + + "text": function( elem ) { + var attr; + return elem.nodeName.toLowerCase() === "input" && + elem.type === "text" && + + // Support: IE<8 + // New HTML5 attribute values (e.g., "search") appear with elem.type === "text" + ( ( attr = elem.getAttribute( "type" ) ) == null || + attr.toLowerCase() === "text" ); + }, + + // Position-in-collection + "first": createPositionalPseudo( function() { + return [ 0 ]; + } ), + + "last": createPositionalPseudo( function( _matchIndexes, length ) { + return [ length - 1 ]; + } ), + + "eq": createPositionalPseudo( function( _matchIndexes, length, argument ) { + return [ argument < 0 ? argument + length : argument ]; + } ), + + "even": createPositionalPseudo( function( matchIndexes, length ) { + var i = 0; + for ( ; i < length; i += 2 ) { + matchIndexes.push( i ); + } + return matchIndexes; + } ), + + "odd": createPositionalPseudo( function( matchIndexes, length ) { + var i = 1; + for ( ; i < length; i += 2 ) { + matchIndexes.push( i ); + } + return matchIndexes; + } ), + + "lt": createPositionalPseudo( function( matchIndexes, length, argument ) { + var i = argument < 0 ? + argument + length : + argument > length ? + length : + argument; + for ( ; --i >= 0; ) { + matchIndexes.push( i ); + } + return matchIndexes; + } ), + + "gt": createPositionalPseudo( function( matchIndexes, length, argument ) { + var i = argument < 0 ? argument + length : argument; + for ( ; ++i < length; ) { + matchIndexes.push( i ); + } + return matchIndexes; + } ) + } +}; + +Expr.pseudos[ "nth" ] = Expr.pseudos[ "eq" ]; + +// Add button/input type pseudos +for ( i in { radio: true, checkbox: true, file: true, password: true, image: true } ) { + Expr.pseudos[ i ] = createInputPseudo( i ); +} +for ( i in { submit: true, reset: true } ) { + Expr.pseudos[ i ] = createButtonPseudo( i ); +} + +// Easy API for creating new setFilters +function setFilters() {} +setFilters.prototype = Expr.filters = Expr.pseudos; +Expr.setFilters = new setFilters(); + +tokenize = Sizzle.tokenize = function( selector, parseOnly ) { + var matched, match, tokens, type, + soFar, groups, preFilters, + cached = tokenCache[ selector + " " ]; + + if ( cached ) { + return parseOnly ? 0 : cached.slice( 0 ); + } + + soFar = selector; + groups = []; + preFilters = Expr.preFilter; + + while ( soFar ) { + + // Comma and first run + if ( !matched || ( match = rcomma.exec( soFar ) ) ) { + if ( match ) { + + // Don't consume trailing commas as valid + soFar = soFar.slice( match[ 0 ].length ) || soFar; + } + groups.push( ( tokens = [] ) ); + } + + matched = false; + + // Combinators + if ( ( match = rcombinators.exec( soFar ) ) ) { + matched = match.shift(); + tokens.push( { + value: matched, + + // Cast descendant combinators to space + type: match[ 0 ].replace( rtrim, " " ) + } ); + soFar = soFar.slice( matched.length ); + } + + // Filters + for ( type in Expr.filter ) { + if ( ( match = matchExpr[ type ].exec( soFar ) ) && ( !preFilters[ type ] || + ( match = preFilters[ type ]( match ) ) ) ) { + matched = match.shift(); + tokens.push( { + value: matched, + type: type, + matches: match + } ); + soFar = soFar.slice( matched.length ); + } + } + + if ( !matched ) { + break; + } + } + + // Return the length of the invalid excess + // if we're just parsing + // Otherwise, throw an error or return tokens + return parseOnly ? + soFar.length : + soFar ? + Sizzle.error( selector ) : + + // Cache the tokens + tokenCache( selector, groups ).slice( 0 ); +}; + +function toSelector( tokens ) { + var i = 0, + len = tokens.length, + selector = ""; + for ( ; i < len; i++ ) { + selector += tokens[ i ].value; + } + return selector; +} + +function addCombinator( matcher, combinator, base ) { + var dir = combinator.dir, + skip = combinator.next, + key = skip || dir, + checkNonElements = base && key === "parentNode", + doneName = done++; + + return combinator.first ? + + // Check against closest ancestor/preceding element + function( elem, context, xml ) { + while ( ( elem = elem[ dir ] ) ) { + if ( elem.nodeType === 1 || checkNonElements ) { + return matcher( elem, context, xml ); + } + } + return false; + } : + + // Check against all ancestor/preceding elements + function( elem, context, xml ) { + var oldCache, uniqueCache, outerCache, + newCache = [ dirruns, doneName ]; + + // We can't set arbitrary data on XML nodes, so they don't benefit from combinator caching + if ( xml ) { + while ( ( elem = elem[ dir ] ) ) { + if ( elem.nodeType === 1 || checkNonElements ) { + if ( matcher( elem, context, xml ) ) { + return true; + } + } + } + } else { + while ( ( elem = elem[ dir ] ) ) { + if ( elem.nodeType === 1 || checkNonElements ) { + outerCache = elem[ expando ] || ( elem[ expando ] = {} ); + + // Support: IE <9 only + // Defend against cloned attroperties (jQuery gh-1709) + uniqueCache = outerCache[ elem.uniqueID ] || + ( outerCache[ elem.uniqueID ] = {} ); + + if ( skip && skip === elem.nodeName.toLowerCase() ) { + elem = elem[ dir ] || elem; + } else if ( ( oldCache = uniqueCache[ key ] ) && + oldCache[ 0 ] === dirruns && oldCache[ 1 ] === doneName ) { + + // Assign to newCache so results back-propagate to previous elements + return ( newCache[ 2 ] = oldCache[ 2 ] ); + } else { + + // Reuse newcache so results back-propagate to previous elements + uniqueCache[ key ] = newCache; + + // A match means we're done; a fail means we have to keep checking + if ( ( newCache[ 2 ] = matcher( elem, context, xml ) ) ) { + return true; + } + } + } + } + } + return false; + }; +} + +function elementMatcher( matchers ) { + return matchers.length > 1 ? + function( elem, context, xml ) { + var i = matchers.length; + while ( i-- ) { + if ( !matchers[ i ]( elem, context, xml ) ) { + return false; + } + } + return true; + } : + matchers[ 0 ]; +} + +function multipleContexts( selector, contexts, results ) { + var i = 0, + len = contexts.length; + for ( ; i < len; i++ ) { + Sizzle( selector, contexts[ i ], results ); + } + return results; +} + +function condense( unmatched, map, filter, context, xml ) { + var elem, + newUnmatched = [], + i = 0, + len = unmatched.length, + mapped = map != null; + + for ( ; i < len; i++ ) { + if ( ( elem = unmatched[ i ] ) ) { + if ( !filter || filter( elem, context, xml ) ) { + newUnmatched.push( elem ); + if ( mapped ) { + map.push( i ); + } + } + } + } + + return newUnmatched; +} + +function setMatcher( preFilter, selector, matcher, postFilter, postFinder, postSelector ) { + if ( postFilter && !postFilter[ expando ] ) { + postFilter = setMatcher( postFilter ); + } + if ( postFinder && !postFinder[ expando ] ) { + postFinder = setMatcher( postFinder, postSelector ); + } + return markFunction( function( seed, results, context, xml ) { + var temp, i, elem, + preMap = [], + postMap = [], + preexisting = results.length, + + // Get initial elements from seed or context + elems = seed || multipleContexts( + selector || "*", + context.nodeType ? [ context ] : context, + [] + ), + + // Prefilter to get matcher input, preserving a map for seed-results synchronization + matcherIn = preFilter && ( seed || !selector ) ? + condense( elems, preMap, preFilter, context, xml ) : + elems, + + matcherOut = matcher ? + + // If we have a postFinder, or filtered seed, or non-seed postFilter or preexisting results, + postFinder || ( seed ? preFilter : preexisting || postFilter ) ? + + // ...intermediate processing is necessary + [] : + + // ...otherwise use results directly + results : + matcherIn; + + // Find primary matches + if ( matcher ) { + matcher( matcherIn, matcherOut, context, xml ); + } + + // Apply postFilter + if ( postFilter ) { + temp = condense( matcherOut, postMap ); + postFilter( temp, [], context, xml ); + + // Un-match failing elements by moving them back to matcherIn + i = temp.length; + while ( i-- ) { + if ( ( elem = temp[ i ] ) ) { + matcherOut[ postMap[ i ] ] = !( matcherIn[ postMap[ i ] ] = elem ); + } + } + } + + if ( seed ) { + if ( postFinder || preFilter ) { + if ( postFinder ) { + + // Get the final matcherOut by condensing this intermediate into postFinder contexts + temp = []; + i = matcherOut.length; + while ( i-- ) { + if ( ( elem = matcherOut[ i ] ) ) { + + // Restore matcherIn since elem is not yet a final match + temp.push( ( matcherIn[ i ] = elem ) ); + } + } + postFinder( null, ( matcherOut = [] ), temp, xml ); + } + + // Move matched elements from seed to results to keep them synchronized + i = matcherOut.length; + while ( i-- ) { + if ( ( elem = matcherOut[ i ] ) && + ( temp = postFinder ? indexOf( seed, elem ) : preMap[ i ] ) > -1 ) { + + seed[ temp ] = !( results[ temp ] = elem ); + } + } + } + + // Add elements to results, through postFinder if defined + } else { + matcherOut = condense( + matcherOut === results ? + matcherOut.splice( preexisting, matcherOut.length ) : + matcherOut + ); + if ( postFinder ) { + postFinder( null, results, matcherOut, xml ); + } else { + push.apply( results, matcherOut ); + } + } + } ); +} + +function matcherFromTokens( tokens ) { + var checkContext, matcher, j, + len = tokens.length, + leadingRelative = Expr.relative[ tokens[ 0 ].type ], + implicitRelative = leadingRelative || Expr.relative[ " " ], + i = leadingRelative ? 1 : 0, + + // The foundational matcher ensures that elements are reachable from top-level context(s) + matchContext = addCombinator( function( elem ) { + return elem === checkContext; + }, implicitRelative, true ), + matchAnyContext = addCombinator( function( elem ) { + return indexOf( checkContext, elem ) > -1; + }, implicitRelative, true ), + matchers = [ function( elem, context, xml ) { + var ret = ( !leadingRelative && ( xml || context !== outermostContext ) ) || ( + ( checkContext = context ).nodeType ? + matchContext( elem, context, xml ) : + matchAnyContext( elem, context, xml ) ); + + // Avoid hanging onto element (issue #299) + checkContext = null; + return ret; + } ]; + + for ( ; i < len; i++ ) { + if ( ( matcher = Expr.relative[ tokens[ i ].type ] ) ) { + matchers = [ addCombinator( elementMatcher( matchers ), matcher ) ]; + } else { + matcher = Expr.filter[ tokens[ i ].type ].apply( null, tokens[ i ].matches ); + + // Return special upon seeing a positional matcher + if ( matcher[ expando ] ) { + + // Find the next relative operator (if any) for proper handling + j = ++i; + for ( ; j < len; j++ ) { + if ( Expr.relative[ tokens[ j ].type ] ) { + break; + } + } + return setMatcher( + i > 1 && elementMatcher( matchers ), + i > 1 && toSelector( + + // If the preceding token was a descendant combinator, insert an implicit any-element `*` + tokens + .slice( 0, i - 1 ) + .concat( { value: tokens[ i - 2 ].type === " " ? "*" : "" } ) + ).replace( rtrim, "$1" ), + matcher, + i < j && matcherFromTokens( tokens.slice( i, j ) ), + j < len && matcherFromTokens( ( tokens = tokens.slice( j ) ) ), + j < len && toSelector( tokens ) + ); + } + matchers.push( matcher ); + } + } + + return elementMatcher( matchers ); +} + +function matcherFromGroupMatchers( elementMatchers, setMatchers ) { + var bySet = setMatchers.length > 0, + byElement = elementMatchers.length > 0, + superMatcher = function( seed, context, xml, results, outermost ) { + var elem, j, matcher, + matchedCount = 0, + i = "0", + unmatched = seed && [], + setMatched = [], + contextBackup = outermostContext, + + // We must always have either seed elements or outermost context + elems = seed || byElement && Expr.find[ "TAG" ]( "*", outermost ), + + // Use integer dirruns iff this is the outermost matcher + dirrunsUnique = ( dirruns += contextBackup == null ? 1 : Math.random() || 0.1 ), + len = elems.length; + + if ( outermost ) { + + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + outermostContext = context == document || context || outermost; + } + + // Add elements passing elementMatchers directly to results + // Support: IE<9, Safari + // Tolerate NodeList properties (IE: "length"; Safari: ) matching elements by id + for ( ; i !== len && ( elem = elems[ i ] ) != null; i++ ) { + if ( byElement && elem ) { + j = 0; + + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + if ( !context && elem.ownerDocument != document ) { + setDocument( elem ); + xml = !documentIsHTML; + } + while ( ( matcher = elementMatchers[ j++ ] ) ) { + if ( matcher( elem, context || document, xml ) ) { + results.push( elem ); + break; + } + } + if ( outermost ) { + dirruns = dirrunsUnique; + } + } + + // Track unmatched elements for set filters + if ( bySet ) { + + // They will have gone through all possible matchers + if ( ( elem = !matcher && elem ) ) { + matchedCount--; + } + + // Lengthen the array for every element, matched or not + if ( seed ) { + unmatched.push( elem ); + } + } + } + + // `i` is now the count of elements visited above, and adding it to `matchedCount` + // makes the latter nonnegative. + matchedCount += i; + + // Apply set filters to unmatched elements + // NOTE: This can be skipped if there are no unmatched elements (i.e., `matchedCount` + // equals `i`), unless we didn't visit _any_ elements in the above loop because we have + // no element matchers and no seed. + // Incrementing an initially-string "0" `i` allows `i` to remain a string only in that + // case, which will result in a "00" `matchedCount` that differs from `i` but is also + // numerically zero. + if ( bySet && i !== matchedCount ) { + j = 0; + while ( ( matcher = setMatchers[ j++ ] ) ) { + matcher( unmatched, setMatched, context, xml ); + } + + if ( seed ) { + + // Reintegrate element matches to eliminate the need for sorting + if ( matchedCount > 0 ) { + while ( i-- ) { + if ( !( unmatched[ i ] || setMatched[ i ] ) ) { + setMatched[ i ] = pop.call( results ); + } + } + } + + // Discard index placeholder values to get only actual matches + setMatched = condense( setMatched ); + } + + // Add matches to results + push.apply( results, setMatched ); + + // Seedless set matches succeeding multiple successful matchers stipulate sorting + if ( outermost && !seed && setMatched.length > 0 && + ( matchedCount + setMatchers.length ) > 1 ) { + + Sizzle.uniqueSort( results ); + } + } + + // Override manipulation of globals by nested matchers + if ( outermost ) { + dirruns = dirrunsUnique; + outermostContext = contextBackup; + } + + return unmatched; + }; + + return bySet ? + markFunction( superMatcher ) : + superMatcher; +} + +compile = Sizzle.compile = function( selector, match /* Internal Use Only */ ) { + var i, + setMatchers = [], + elementMatchers = [], + cached = compilerCache[ selector + " " ]; + + if ( !cached ) { + + // Generate a function of recursive functions that can be used to check each element + if ( !match ) { + match = tokenize( selector ); + } + i = match.length; + while ( i-- ) { + cached = matcherFromTokens( match[ i ] ); + if ( cached[ expando ] ) { + setMatchers.push( cached ); + } else { + elementMatchers.push( cached ); + } + } + + // Cache the compiled function + cached = compilerCache( + selector, + matcherFromGroupMatchers( elementMatchers, setMatchers ) + ); + + // Save selector and tokenization + cached.selector = selector; + } + return cached; +}; + +/** + * A low-level selection function that works with Sizzle's compiled + * selector functions + * @param {String|Function} selector A selector or a pre-compiled + * selector function built with Sizzle.compile + * @param {Element} context + * @param {Array} [results] + * @param {Array} [seed] A set of elements to match against + */ +select = Sizzle.select = function( selector, context, results, seed ) { + var i, tokens, token, type, find, + compiled = typeof selector === "function" && selector, + match = !seed && tokenize( ( selector = compiled.selector || selector ) ); + + results = results || []; + + // Try to minimize operations if there is only one selector in the list and no seed + // (the latter of which guarantees us context) + if ( match.length === 1 ) { + + // Reduce context if the leading compound selector is an ID + tokens = match[ 0 ] = match[ 0 ].slice( 0 ); + if ( tokens.length > 2 && ( token = tokens[ 0 ] ).type === "ID" && + context.nodeType === 9 && documentIsHTML && Expr.relative[ tokens[ 1 ].type ] ) { + + context = ( Expr.find[ "ID" ]( token.matches[ 0 ] + .replace( runescape, funescape ), context ) || [] )[ 0 ]; + if ( !context ) { + return results; + + // Precompiled matchers will still verify ancestry, so step up a level + } else if ( compiled ) { + context = context.parentNode; + } + + selector = selector.slice( tokens.shift().value.length ); + } + + // Fetch a seed set for right-to-left matching + i = matchExpr[ "needsContext" ].test( selector ) ? 0 : tokens.length; + while ( i-- ) { + token = tokens[ i ]; + + // Abort if we hit a combinator + if ( Expr.relative[ ( type = token.type ) ] ) { + break; + } + if ( ( find = Expr.find[ type ] ) ) { + + // Search, expanding context for leading sibling combinators + if ( ( seed = find( + token.matches[ 0 ].replace( runescape, funescape ), + rsibling.test( tokens[ 0 ].type ) && testContext( context.parentNode ) || + context + ) ) ) { + + // If seed is empty or no tokens remain, we can return early + tokens.splice( i, 1 ); + selector = seed.length && toSelector( tokens ); + if ( !selector ) { + push.apply( results, seed ); + return results; + } + + break; + } + } + } + } + + // Compile and execute a filtering function if one is not provided + // Provide `match` to avoid retokenization if we modified the selector above + ( compiled || compile( selector, match ) )( + seed, + context, + !documentIsHTML, + results, + !context || rsibling.test( selector ) && testContext( context.parentNode ) || context + ); + return results; +}; + +// One-time assignments + +// Sort stability +support.sortStable = expando.split( "" ).sort( sortOrder ).join( "" ) === expando; + +// Support: Chrome 14-35+ +// Always assume duplicates if they aren't passed to the comparison function +support.detectDuplicates = !!hasDuplicate; + +// Initialize against the default document +setDocument(); + +// Support: Webkit<537.32 - Safari 6.0.3/Chrome 25 (fixed in Chrome 27) +// Detached nodes confoundingly follow *each other* +support.sortDetached = assert( function( el ) { + + // Should return 1, but returns 4 (following) + return el.compareDocumentPosition( document.createElement( "fieldset" ) ) & 1; +} ); + +// Support: IE<8 +// Prevent attribute/property "interpolation" +// https://msdn.microsoft.com/en-us/library/ms536429%28VS.85%29.aspx +if ( !assert( function( el ) { + el.innerHTML = ""; + return el.firstChild.getAttribute( "href" ) === "#"; +} ) ) { + addHandle( "type|href|height|width", function( elem, name, isXML ) { + if ( !isXML ) { + return elem.getAttribute( name, name.toLowerCase() === "type" ? 1 : 2 ); + } + } ); +} + +// Support: IE<9 +// Use defaultValue in place of getAttribute("value") +if ( !support.attributes || !assert( function( el ) { + el.innerHTML = ""; + el.firstChild.setAttribute( "value", "" ); + return el.firstChild.getAttribute( "value" ) === ""; +} ) ) { + addHandle( "value", function( elem, _name, isXML ) { + if ( !isXML && elem.nodeName.toLowerCase() === "input" ) { + return elem.defaultValue; + } + } ); +} + +// Support: IE<9 +// Use getAttributeNode to fetch booleans when getAttribute lies +if ( !assert( function( el ) { + return el.getAttribute( "disabled" ) == null; +} ) ) { + addHandle( booleans, function( elem, name, isXML ) { + var val; + if ( !isXML ) { + return elem[ name ] === true ? name.toLowerCase() : + ( val = elem.getAttributeNode( name ) ) && val.specified ? + val.value : + null; + } + } ); +} + +return Sizzle; + +} )( window ); + + + +jQuery.find = Sizzle; +jQuery.expr = Sizzle.selectors; + +// Deprecated +jQuery.expr[ ":" ] = jQuery.expr.pseudos; +jQuery.uniqueSort = jQuery.unique = Sizzle.uniqueSort; +jQuery.text = Sizzle.getText; +jQuery.isXMLDoc = Sizzle.isXML; +jQuery.contains = Sizzle.contains; +jQuery.escapeSelector = Sizzle.escape; + + + + +var dir = function( elem, dir, until ) { + var matched = [], + truncate = until !== undefined; + + while ( ( elem = elem[ dir ] ) && elem.nodeType !== 9 ) { + if ( elem.nodeType === 1 ) { + if ( truncate && jQuery( elem ).is( until ) ) { + break; + } + matched.push( elem ); + } + } + return matched; +}; + + +var siblings = function( n, elem ) { + var matched = []; + + for ( ; n; n = n.nextSibling ) { + if ( n.nodeType === 1 && n !== elem ) { + matched.push( n ); + } + } + + return matched; +}; + + +var rneedsContext = jQuery.expr.match.needsContext; + + + +function nodeName( elem, name ) { + + return elem.nodeName && elem.nodeName.toLowerCase() === name.toLowerCase(); + +} +var rsingleTag = ( /^<([a-z][^\/\0>:\x20\t\r\n\f]*)[\x20\t\r\n\f]*\/?>(?:<\/\1>|)$/i ); + + + +// Implement the identical functionality for filter and not +function winnow( elements, qualifier, not ) { + if ( isFunction( qualifier ) ) { + return jQuery.grep( elements, function( elem, i ) { + return !!qualifier.call( elem, i, elem ) !== not; + } ); + } + + // Single element + if ( qualifier.nodeType ) { + return jQuery.grep( elements, function( elem ) { + return ( elem === qualifier ) !== not; + } ); + } + + // Arraylike of elements (jQuery, arguments, Array) + if ( typeof qualifier !== "string" ) { + return jQuery.grep( elements, function( elem ) { + return ( indexOf.call( qualifier, elem ) > -1 ) !== not; + } ); + } + + // Filtered directly for both simple and complex selectors + return jQuery.filter( qualifier, elements, not ); +} + +jQuery.filter = function( expr, elems, not ) { + var elem = elems[ 0 ]; + + if ( not ) { + expr = ":not(" + expr + ")"; + } + + if ( elems.length === 1 && elem.nodeType === 1 ) { + return jQuery.find.matchesSelector( elem, expr ) ? [ elem ] : []; + } + + return jQuery.find.matches( expr, jQuery.grep( elems, function( elem ) { + return elem.nodeType === 1; + } ) ); +}; + +jQuery.fn.extend( { + find: function( selector ) { + var i, ret, + len = this.length, + self = this; + + if ( typeof selector !== "string" ) { + return this.pushStack( jQuery( selector ).filter( function() { + for ( i = 0; i < len; i++ ) { + if ( jQuery.contains( self[ i ], this ) ) { + return true; + } + } + } ) ); + } + + ret = this.pushStack( [] ); + + for ( i = 0; i < len; i++ ) { + jQuery.find( selector, self[ i ], ret ); + } + + return len > 1 ? jQuery.uniqueSort( ret ) : ret; + }, + filter: function( selector ) { + return this.pushStack( winnow( this, selector || [], false ) ); + }, + not: function( selector ) { + return this.pushStack( winnow( this, selector || [], true ) ); + }, + is: function( selector ) { + return !!winnow( + this, + + // If this is a positional/relative selector, check membership in the returned set + // so $("p:first").is("p:last") won't return true for a doc with two "p". + typeof selector === "string" && rneedsContext.test( selector ) ? + jQuery( selector ) : + selector || [], + false + ).length; + } +} ); + + +// Initialize a jQuery object + + +// A central reference to the root jQuery(document) +var rootjQuery, + + // A simple way to check for HTML strings + // Prioritize #id over to avoid XSS via location.hash (#9521) + // Strict HTML recognition (#11290: must start with <) + // Shortcut simple #id case for speed + rquickExpr = /^(?:\s*(<[\w\W]+>)[^>]*|#([\w-]+))$/, + + init = jQuery.fn.init = function( selector, context, root ) { + var match, elem; + + // HANDLE: $(""), $(null), $(undefined), $(false) + if ( !selector ) { + return this; + } + + // Method init() accepts an alternate rootjQuery + // so migrate can support jQuery.sub (gh-2101) + root = root || rootjQuery; + + // Handle HTML strings + if ( typeof selector === "string" ) { + if ( selector[ 0 ] === "<" && + selector[ selector.length - 1 ] === ">" && + selector.length >= 3 ) { + + // Assume that strings that start and end with <> are HTML and skip the regex check + match = [ null, selector, null ]; + + } else { + match = rquickExpr.exec( selector ); + } + + // Match html or make sure no context is specified for #id + if ( match && ( match[ 1 ] || !context ) ) { + + // HANDLE: $(html) -> $(array) + if ( match[ 1 ] ) { + context = context instanceof jQuery ? context[ 0 ] : context; + + // Option to run scripts is true for back-compat + // Intentionally let the error be thrown if parseHTML is not present + jQuery.merge( this, jQuery.parseHTML( + match[ 1 ], + context && context.nodeType ? context.ownerDocument || context : document, + true + ) ); + + // HANDLE: $(html, props) + if ( rsingleTag.test( match[ 1 ] ) && jQuery.isPlainObject( context ) ) { + for ( match in context ) { + + // Properties of context are called as methods if possible + if ( isFunction( this[ match ] ) ) { + this[ match ]( context[ match ] ); + + // ...and otherwise set as attributes + } else { + this.attr( match, context[ match ] ); + } + } + } + + return this; + + // HANDLE: $(#id) + } else { + elem = document.getElementById( match[ 2 ] ); + + if ( elem ) { + + // Inject the element directly into the jQuery object + this[ 0 ] = elem; + this.length = 1; + } + return this; + } + + // HANDLE: $(expr, $(...)) + } else if ( !context || context.jquery ) { + return ( context || root ).find( selector ); + + // HANDLE: $(expr, context) + // (which is just equivalent to: $(context).find(expr) + } else { + return this.constructor( context ).find( selector ); + } + + // HANDLE: $(DOMElement) + } else if ( selector.nodeType ) { + this[ 0 ] = selector; + this.length = 1; + return this; + + // HANDLE: $(function) + // Shortcut for document ready + } else if ( isFunction( selector ) ) { + return root.ready !== undefined ? + root.ready( selector ) : + + // Execute immediately if ready is not present + selector( jQuery ); + } + + return jQuery.makeArray( selector, this ); + }; + +// Give the init function the jQuery prototype for later instantiation +init.prototype = jQuery.fn; + +// Initialize central reference +rootjQuery = jQuery( document ); + + +var rparentsprev = /^(?:parents|prev(?:Until|All))/, + + // Methods guaranteed to produce a unique set when starting from a unique set + guaranteedUnique = { + children: true, + contents: true, + next: true, + prev: true + }; + +jQuery.fn.extend( { + has: function( target ) { + var targets = jQuery( target, this ), + l = targets.length; + + return this.filter( function() { + var i = 0; + for ( ; i < l; i++ ) { + if ( jQuery.contains( this, targets[ i ] ) ) { + return true; + } + } + } ); + }, + + closest: function( selectors, context ) { + var cur, + i = 0, + l = this.length, + matched = [], + targets = typeof selectors !== "string" && jQuery( selectors ); + + // Positional selectors never match, since there's no _selection_ context + if ( !rneedsContext.test( selectors ) ) { + for ( ; i < l; i++ ) { + for ( cur = this[ i ]; cur && cur !== context; cur = cur.parentNode ) { + + // Always skip document fragments + if ( cur.nodeType < 11 && ( targets ? + targets.index( cur ) > -1 : + + // Don't pass non-elements to Sizzle + cur.nodeType === 1 && + jQuery.find.matchesSelector( cur, selectors ) ) ) { + + matched.push( cur ); + break; + } + } + } + } + + return this.pushStack( matched.length > 1 ? jQuery.uniqueSort( matched ) : matched ); + }, + + // Determine the position of an element within the set + index: function( elem ) { + + // No argument, return index in parent + if ( !elem ) { + return ( this[ 0 ] && this[ 0 ].parentNode ) ? this.first().prevAll().length : -1; + } + + // Index in selector + if ( typeof elem === "string" ) { + return indexOf.call( jQuery( elem ), this[ 0 ] ); + } + + // Locate the position of the desired element + return indexOf.call( this, + + // If it receives a jQuery object, the first element is used + elem.jquery ? elem[ 0 ] : elem + ); + }, + + add: function( selector, context ) { + return this.pushStack( + jQuery.uniqueSort( + jQuery.merge( this.get(), jQuery( selector, context ) ) + ) + ); + }, + + addBack: function( selector ) { + return this.add( selector == null ? + this.prevObject : this.prevObject.filter( selector ) + ); + } +} ); + +function sibling( cur, dir ) { + while ( ( cur = cur[ dir ] ) && cur.nodeType !== 1 ) {} + return cur; +} + +jQuery.each( { + parent: function( elem ) { + var parent = elem.parentNode; + return parent && parent.nodeType !== 11 ? parent : null; + }, + parents: function( elem ) { + return dir( elem, "parentNode" ); + }, + parentsUntil: function( elem, _i, until ) { + return dir( elem, "parentNode", until ); + }, + next: function( elem ) { + return sibling( elem, "nextSibling" ); + }, + prev: function( elem ) { + return sibling( elem, "previousSibling" ); + }, + nextAll: function( elem ) { + return dir( elem, "nextSibling" ); + }, + prevAll: function( elem ) { + return dir( elem, "previousSibling" ); + }, + nextUntil: function( elem, _i, until ) { + return dir( elem, "nextSibling", until ); + }, + prevUntil: function( elem, _i, until ) { + return dir( elem, "previousSibling", until ); + }, + siblings: function( elem ) { + return siblings( ( elem.parentNode || {} ).firstChild, elem ); + }, + children: function( elem ) { + return siblings( elem.firstChild ); + }, + contents: function( elem ) { + if ( elem.contentDocument != null && + + // Support: IE 11+ + // elements with no `data` attribute has an object + // `contentDocument` with a `null` prototype. + getProto( elem.contentDocument ) ) { + + return elem.contentDocument; + } + + // Support: IE 9 - 11 only, iOS 7 only, Android Browser <=4.3 only + // Treat the template element as a regular one in browsers that + // don't support it. + if ( nodeName( elem, "template" ) ) { + elem = elem.content || elem; + } + + return jQuery.merge( [], elem.childNodes ); + } +}, function( name, fn ) { + jQuery.fn[ name ] = function( until, selector ) { + var matched = jQuery.map( this, fn, until ); + + if ( name.slice( -5 ) !== "Until" ) { + selector = until; + } + + if ( selector && typeof selector === "string" ) { + matched = jQuery.filter( selector, matched ); + } + + if ( this.length > 1 ) { + + // Remove duplicates + if ( !guaranteedUnique[ name ] ) { + jQuery.uniqueSort( matched ); + } + + // Reverse order for parents* and prev-derivatives + if ( rparentsprev.test( name ) ) { + matched.reverse(); + } + } + + return this.pushStack( matched ); + }; +} ); +var rnothtmlwhite = ( /[^\x20\t\r\n\f]+/g ); + + + +// Convert String-formatted options into Object-formatted ones +function createOptions( options ) { + var object = {}; + jQuery.each( options.match( rnothtmlwhite ) || [], function( _, flag ) { + object[ flag ] = true; + } ); + return object; +} + +/* + * Create a callback list using the following parameters: + * + * options: an optional list of space-separated options that will change how + * the callback list behaves or a more traditional option object + * + * By default a callback list will act like an event callback list and can be + * "fired" multiple times. + * + * Possible options: + * + * once: will ensure the callback list can only be fired once (like a Deferred) + * + * memory: will keep track of previous values and will call any callback added + * after the list has been fired right away with the latest "memorized" + * values (like a Deferred) + * + * unique: will ensure a callback can only be added once (no duplicate in the list) + * + * stopOnFalse: interrupt callings when a callback returns false + * + */ +jQuery.Callbacks = function( options ) { + + // Convert options from String-formatted to Object-formatted if needed + // (we check in cache first) + options = typeof options === "string" ? + createOptions( options ) : + jQuery.extend( {}, options ); + + var // Flag to know if list is currently firing + firing, + + // Last fire value for non-forgettable lists + memory, + + // Flag to know if list was already fired + fired, + + // Flag to prevent firing + locked, + + // Actual callback list + list = [], + + // Queue of execution data for repeatable lists + queue = [], + + // Index of currently firing callback (modified by add/remove as needed) + firingIndex = -1, + + // Fire callbacks + fire = function() { + + // Enforce single-firing + locked = locked || options.once; + + // Execute callbacks for all pending executions, + // respecting firingIndex overrides and runtime changes + fired = firing = true; + for ( ; queue.length; firingIndex = -1 ) { + memory = queue.shift(); + while ( ++firingIndex < list.length ) { + + // Run callback and check for early termination + if ( list[ firingIndex ].apply( memory[ 0 ], memory[ 1 ] ) === false && + options.stopOnFalse ) { + + // Jump to end and forget the data so .add doesn't re-fire + firingIndex = list.length; + memory = false; + } + } + } + + // Forget the data if we're done with it + if ( !options.memory ) { + memory = false; + } + + firing = false; + + // Clean up if we're done firing for good + if ( locked ) { + + // Keep an empty list if we have data for future add calls + if ( memory ) { + list = []; + + // Otherwise, this object is spent + } else { + list = ""; + } + } + }, + + // Actual Callbacks object + self = { + + // Add a callback or a collection of callbacks to the list + add: function() { + if ( list ) { + + // If we have memory from a past run, we should fire after adding + if ( memory && !firing ) { + firingIndex = list.length - 1; + queue.push( memory ); + } + + ( function add( args ) { + jQuery.each( args, function( _, arg ) { + if ( isFunction( arg ) ) { + if ( !options.unique || !self.has( arg ) ) { + list.push( arg ); + } + } else if ( arg && arg.length && toType( arg ) !== "string" ) { + + // Inspect recursively + add( arg ); + } + } ); + } )( arguments ); + + if ( memory && !firing ) { + fire(); + } + } + return this; + }, + + // Remove a callback from the list + remove: function() { + jQuery.each( arguments, function( _, arg ) { + var index; + while ( ( index = jQuery.inArray( arg, list, index ) ) > -1 ) { + list.splice( index, 1 ); + + // Handle firing indexes + if ( index <= firingIndex ) { + firingIndex--; + } + } + } ); + return this; + }, + + // Check if a given callback is in the list. + // If no argument is given, return whether or not list has callbacks attached. + has: function( fn ) { + return fn ? + jQuery.inArray( fn, list ) > -1 : + list.length > 0; + }, + + // Remove all callbacks from the list + empty: function() { + if ( list ) { + list = []; + } + return this; + }, + + // Disable .fire and .add + // Abort any current/pending executions + // Clear all callbacks and values + disable: function() { + locked = queue = []; + list = memory = ""; + return this; + }, + disabled: function() { + return !list; + }, + + // Disable .fire + // Also disable .add unless we have memory (since it would have no effect) + // Abort any pending executions + lock: function() { + locked = queue = []; + if ( !memory && !firing ) { + list = memory = ""; + } + return this; + }, + locked: function() { + return !!locked; + }, + + // Call all callbacks with the given context and arguments + fireWith: function( context, args ) { + if ( !locked ) { + args = args || []; + args = [ context, args.slice ? args.slice() : args ]; + queue.push( args ); + if ( !firing ) { + fire(); + } + } + return this; + }, + + // Call all the callbacks with the given arguments + fire: function() { + self.fireWith( this, arguments ); + return this; + }, + + // To know if the callbacks have already been called at least once + fired: function() { + return !!fired; + } + }; + + return self; +}; + + +function Identity( v ) { + return v; +} +function Thrower( ex ) { + throw ex; +} + +function adoptValue( value, resolve, reject, noValue ) { + var method; + + try { + + // Check for promise aspect first to privilege synchronous behavior + if ( value && isFunction( ( method = value.promise ) ) ) { + method.call( value ).done( resolve ).fail( reject ); + + // Other thenables + } else if ( value && isFunction( ( method = value.then ) ) ) { + method.call( value, resolve, reject ); + + // Other non-thenables + } else { + + // Control `resolve` arguments by letting Array#slice cast boolean `noValue` to integer: + // * false: [ value ].slice( 0 ) => resolve( value ) + // * true: [ value ].slice( 1 ) => resolve() + resolve.apply( undefined, [ value ].slice( noValue ) ); + } + + // For Promises/A+, convert exceptions into rejections + // Since jQuery.when doesn't unwrap thenables, we can skip the extra checks appearing in + // Deferred#then to conditionally suppress rejection. + } catch ( value ) { + + // Support: Android 4.0 only + // Strict mode functions invoked without .call/.apply get global-object context + reject.apply( undefined, [ value ] ); + } +} + +jQuery.extend( { + + Deferred: function( func ) { + var tuples = [ + + // action, add listener, callbacks, + // ... .then handlers, argument index, [final state] + [ "notify", "progress", jQuery.Callbacks( "memory" ), + jQuery.Callbacks( "memory" ), 2 ], + [ "resolve", "done", jQuery.Callbacks( "once memory" ), + jQuery.Callbacks( "once memory" ), 0, "resolved" ], + [ "reject", "fail", jQuery.Callbacks( "once memory" ), + jQuery.Callbacks( "once memory" ), 1, "rejected" ] + ], + state = "pending", + promise = { + state: function() { + return state; + }, + always: function() { + deferred.done( arguments ).fail( arguments ); + return this; + }, + "catch": function( fn ) { + return promise.then( null, fn ); + }, + + // Keep pipe for back-compat + pipe: function( /* fnDone, fnFail, fnProgress */ ) { + var fns = arguments; + + return jQuery.Deferred( function( newDefer ) { + jQuery.each( tuples, function( _i, tuple ) { + + // Map tuples (progress, done, fail) to arguments (done, fail, progress) + var fn = isFunction( fns[ tuple[ 4 ] ] ) && fns[ tuple[ 4 ] ]; + + // deferred.progress(function() { bind to newDefer or newDefer.notify }) + // deferred.done(function() { bind to newDefer or newDefer.resolve }) + // deferred.fail(function() { bind to newDefer or newDefer.reject }) + deferred[ tuple[ 1 ] ]( function() { + var returned = fn && fn.apply( this, arguments ); + if ( returned && isFunction( returned.promise ) ) { + returned.promise() + .progress( newDefer.notify ) + .done( newDefer.resolve ) + .fail( newDefer.reject ); + } else { + newDefer[ tuple[ 0 ] + "With" ]( + this, + fn ? [ returned ] : arguments + ); + } + } ); + } ); + fns = null; + } ).promise(); + }, + then: function( onFulfilled, onRejected, onProgress ) { + var maxDepth = 0; + function resolve( depth, deferred, handler, special ) { + return function() { + var that = this, + args = arguments, + mightThrow = function() { + var returned, then; + + // Support: Promises/A+ section 2.3.3.3.3 + // https://promisesaplus.com/#point-59 + // Ignore double-resolution attempts + if ( depth < maxDepth ) { + return; + } + + returned = handler.apply( that, args ); + + // Support: Promises/A+ section 2.3.1 + // https://promisesaplus.com/#point-48 + if ( returned === deferred.promise() ) { + throw new TypeError( "Thenable self-resolution" ); + } + + // Support: Promises/A+ sections 2.3.3.1, 3.5 + // https://promisesaplus.com/#point-54 + // https://promisesaplus.com/#point-75 + // Retrieve `then` only once + then = returned && + + // Support: Promises/A+ section 2.3.4 + // https://promisesaplus.com/#point-64 + // Only check objects and functions for thenability + ( typeof returned === "object" || + typeof returned === "function" ) && + returned.then; + + // Handle a returned thenable + if ( isFunction( then ) ) { + + // Special processors (notify) just wait for resolution + if ( special ) { + then.call( + returned, + resolve( maxDepth, deferred, Identity, special ), + resolve( maxDepth, deferred, Thrower, special ) + ); + + // Normal processors (resolve) also hook into progress + } else { + + // ...and disregard older resolution values + maxDepth++; + + then.call( + returned, + resolve( maxDepth, deferred, Identity, special ), + resolve( maxDepth, deferred, Thrower, special ), + resolve( maxDepth, deferred, Identity, + deferred.notifyWith ) + ); + } + + // Handle all other returned values + } else { + + // Only substitute handlers pass on context + // and multiple values (non-spec behavior) + if ( handler !== Identity ) { + that = undefined; + args = [ returned ]; + } + + // Process the value(s) + // Default process is resolve + ( special || deferred.resolveWith )( that, args ); + } + }, + + // Only normal processors (resolve) catch and reject exceptions + process = special ? + mightThrow : + function() { + try { + mightThrow(); + } catch ( e ) { + + if ( jQuery.Deferred.exceptionHook ) { + jQuery.Deferred.exceptionHook( e, + process.stackTrace ); + } + + // Support: Promises/A+ section 2.3.3.3.4.1 + // https://promisesaplus.com/#point-61 + // Ignore post-resolution exceptions + if ( depth + 1 >= maxDepth ) { + + // Only substitute handlers pass on context + // and multiple values (non-spec behavior) + if ( handler !== Thrower ) { + that = undefined; + args = [ e ]; + } + + deferred.rejectWith( that, args ); + } + } + }; + + // Support: Promises/A+ section 2.3.3.3.1 + // https://promisesaplus.com/#point-57 + // Re-resolve promises immediately to dodge false rejection from + // subsequent errors + if ( depth ) { + process(); + } else { + + // Call an optional hook to record the stack, in case of exception + // since it's otherwise lost when execution goes async + if ( jQuery.Deferred.getStackHook ) { + process.stackTrace = jQuery.Deferred.getStackHook(); + } + window.setTimeout( process ); + } + }; + } + + return jQuery.Deferred( function( newDefer ) { + + // progress_handlers.add( ... ) + tuples[ 0 ][ 3 ].add( + resolve( + 0, + newDefer, + isFunction( onProgress ) ? + onProgress : + Identity, + newDefer.notifyWith + ) + ); + + // fulfilled_handlers.add( ... ) + tuples[ 1 ][ 3 ].add( + resolve( + 0, + newDefer, + isFunction( onFulfilled ) ? + onFulfilled : + Identity + ) + ); + + // rejected_handlers.add( ... ) + tuples[ 2 ][ 3 ].add( + resolve( + 0, + newDefer, + isFunction( onRejected ) ? + onRejected : + Thrower + ) + ); + } ).promise(); + }, + + // Get a promise for this deferred + // If obj is provided, the promise aspect is added to the object + promise: function( obj ) { + return obj != null ? jQuery.extend( obj, promise ) : promise; + } + }, + deferred = {}; + + // Add list-specific methods + jQuery.each( tuples, function( i, tuple ) { + var list = tuple[ 2 ], + stateString = tuple[ 5 ]; + + // promise.progress = list.add + // promise.done = list.add + // promise.fail = list.add + promise[ tuple[ 1 ] ] = list.add; + + // Handle state + if ( stateString ) { + list.add( + function() { + + // state = "resolved" (i.e., fulfilled) + // state = "rejected" + state = stateString; + }, + + // rejected_callbacks.disable + // fulfilled_callbacks.disable + tuples[ 3 - i ][ 2 ].disable, + + // rejected_handlers.disable + // fulfilled_handlers.disable + tuples[ 3 - i ][ 3 ].disable, + + // progress_callbacks.lock + tuples[ 0 ][ 2 ].lock, + + // progress_handlers.lock + tuples[ 0 ][ 3 ].lock + ); + } + + // progress_handlers.fire + // fulfilled_handlers.fire + // rejected_handlers.fire + list.add( tuple[ 3 ].fire ); + + // deferred.notify = function() { deferred.notifyWith(...) } + // deferred.resolve = function() { deferred.resolveWith(...) } + // deferred.reject = function() { deferred.rejectWith(...) } + deferred[ tuple[ 0 ] ] = function() { + deferred[ tuple[ 0 ] + "With" ]( this === deferred ? undefined : this, arguments ); + return this; + }; + + // deferred.notifyWith = list.fireWith + // deferred.resolveWith = list.fireWith + // deferred.rejectWith = list.fireWith + deferred[ tuple[ 0 ] + "With" ] = list.fireWith; + } ); + + // Make the deferred a promise + promise.promise( deferred ); + + // Call given func if any + if ( func ) { + func.call( deferred, deferred ); + } + + // All done! + return deferred; + }, + + // Deferred helper + when: function( singleValue ) { + var + + // count of uncompleted subordinates + remaining = arguments.length, + + // count of unprocessed arguments + i = remaining, + + // subordinate fulfillment data + resolveContexts = Array( i ), + resolveValues = slice.call( arguments ), + + // the primary Deferred + primary = jQuery.Deferred(), + + // subordinate callback factory + updateFunc = function( i ) { + return function( value ) { + resolveContexts[ i ] = this; + resolveValues[ i ] = arguments.length > 1 ? slice.call( arguments ) : value; + if ( !( --remaining ) ) { + primary.resolveWith( resolveContexts, resolveValues ); + } + }; + }; + + // Single- and empty arguments are adopted like Promise.resolve + if ( remaining <= 1 ) { + adoptValue( singleValue, primary.done( updateFunc( i ) ).resolve, primary.reject, + !remaining ); + + // Use .then() to unwrap secondary thenables (cf. gh-3000) + if ( primary.state() === "pending" || + isFunction( resolveValues[ i ] && resolveValues[ i ].then ) ) { + + return primary.then(); + } + } + + // Multiple arguments are aggregated like Promise.all array elements + while ( i-- ) { + adoptValue( resolveValues[ i ], updateFunc( i ), primary.reject ); + } + + return primary.promise(); + } +} ); + + +// These usually indicate a programmer mistake during development, +// warn about them ASAP rather than swallowing them by default. +var rerrorNames = /^(Eval|Internal|Range|Reference|Syntax|Type|URI)Error$/; + +jQuery.Deferred.exceptionHook = function( error, stack ) { + + // Support: IE 8 - 9 only + // Console exists when dev tools are open, which can happen at any time + if ( window.console && window.console.warn && error && rerrorNames.test( error.name ) ) { + window.console.warn( "jQuery.Deferred exception: " + error.message, error.stack, stack ); + } +}; + + + + +jQuery.readyException = function( error ) { + window.setTimeout( function() { + throw error; + } ); +}; + + + + +// The deferred used on DOM ready +var readyList = jQuery.Deferred(); + +jQuery.fn.ready = function( fn ) { + + readyList + .then( fn ) + + // Wrap jQuery.readyException in a function so that the lookup + // happens at the time of error handling instead of callback + // registration. + .catch( function( error ) { + jQuery.readyException( error ); + } ); + + return this; +}; + +jQuery.extend( { + + // Is the DOM ready to be used? Set to true once it occurs. + isReady: false, + + // A counter to track how many items to wait for before + // the ready event fires. See #6781 + readyWait: 1, + + // Handle when the DOM is ready + ready: function( wait ) { + + // Abort if there are pending holds or we're already ready + if ( wait === true ? --jQuery.readyWait : jQuery.isReady ) { + return; + } + + // Remember that the DOM is ready + jQuery.isReady = true; + + // If a normal DOM Ready event fired, decrement, and wait if need be + if ( wait !== true && --jQuery.readyWait > 0 ) { + return; + } + + // If there are functions bound, to execute + readyList.resolveWith( document, [ jQuery ] ); + } +} ); + +jQuery.ready.then = readyList.then; + +// The ready event handler and self cleanup method +function completed() { + document.removeEventListener( "DOMContentLoaded", completed ); + window.removeEventListener( "load", completed ); + jQuery.ready(); +} + +// Catch cases where $(document).ready() is called +// after the browser event has already occurred. +// Support: IE <=9 - 10 only +// Older IE sometimes signals "interactive" too soon +if ( document.readyState === "complete" || + ( document.readyState !== "loading" && !document.documentElement.doScroll ) ) { + + // Handle it asynchronously to allow scripts the opportunity to delay ready + window.setTimeout( jQuery.ready ); + +} else { + + // Use the handy event callback + document.addEventListener( "DOMContentLoaded", completed ); + + // A fallback to window.onload, that will always work + window.addEventListener( "load", completed ); +} + + + + +// Multifunctional method to get and set values of a collection +// The value/s can optionally be executed if it's a function +var access = function( elems, fn, key, value, chainable, emptyGet, raw ) { + var i = 0, + len = elems.length, + bulk = key == null; + + // Sets many values + if ( toType( key ) === "object" ) { + chainable = true; + for ( i in key ) { + access( elems, fn, i, key[ i ], true, emptyGet, raw ); + } + + // Sets one value + } else if ( value !== undefined ) { + chainable = true; + + if ( !isFunction( value ) ) { + raw = true; + } + + if ( bulk ) { + + // Bulk operations run against the entire set + if ( raw ) { + fn.call( elems, value ); + fn = null; + + // ...except when executing function values + } else { + bulk = fn; + fn = function( elem, _key, value ) { + return bulk.call( jQuery( elem ), value ); + }; + } + } + + if ( fn ) { + for ( ; i < len; i++ ) { + fn( + elems[ i ], key, raw ? + value : + value.call( elems[ i ], i, fn( elems[ i ], key ) ) + ); + } + } + } + + if ( chainable ) { + return elems; + } + + // Gets + if ( bulk ) { + return fn.call( elems ); + } + + return len ? fn( elems[ 0 ], key ) : emptyGet; +}; + + +// Matches dashed string for camelizing +var rmsPrefix = /^-ms-/, + rdashAlpha = /-([a-z])/g; + +// Used by camelCase as callback to replace() +function fcamelCase( _all, letter ) { + return letter.toUpperCase(); +} + +// Convert dashed to camelCase; used by the css and data modules +// Support: IE <=9 - 11, Edge 12 - 15 +// Microsoft forgot to hump their vendor prefix (#9572) +function camelCase( string ) { + return string.replace( rmsPrefix, "ms-" ).replace( rdashAlpha, fcamelCase ); +} +var acceptData = function( owner ) { + + // Accepts only: + // - Node + // - Node.ELEMENT_NODE + // - Node.DOCUMENT_NODE + // - Object + // - Any + return owner.nodeType === 1 || owner.nodeType === 9 || !( +owner.nodeType ); +}; + + + + +function Data() { + this.expando = jQuery.expando + Data.uid++; +} + +Data.uid = 1; + +Data.prototype = { + + cache: function( owner ) { + + // Check if the owner object already has a cache + var value = owner[ this.expando ]; + + // If not, create one + if ( !value ) { + value = {}; + + // We can accept data for non-element nodes in modern browsers, + // but we should not, see #8335. + // Always return an empty object. + if ( acceptData( owner ) ) { + + // If it is a node unlikely to be stringify-ed or looped over + // use plain assignment + if ( owner.nodeType ) { + owner[ this.expando ] = value; + + // Otherwise secure it in a non-enumerable property + // configurable must be true to allow the property to be + // deleted when data is removed + } else { + Object.defineProperty( owner, this.expando, { + value: value, + configurable: true + } ); + } + } + } + + return value; + }, + set: function( owner, data, value ) { + var prop, + cache = this.cache( owner ); + + // Handle: [ owner, key, value ] args + // Always use camelCase key (gh-2257) + if ( typeof data === "string" ) { + cache[ camelCase( data ) ] = value; + + // Handle: [ owner, { properties } ] args + } else { + + // Copy the properties one-by-one to the cache object + for ( prop in data ) { + cache[ camelCase( prop ) ] = data[ prop ]; + } + } + return cache; + }, + get: function( owner, key ) { + return key === undefined ? + this.cache( owner ) : + + // Always use camelCase key (gh-2257) + owner[ this.expando ] && owner[ this.expando ][ camelCase( key ) ]; + }, + access: function( owner, key, value ) { + + // In cases where either: + // + // 1. No key was specified + // 2. A string key was specified, but no value provided + // + // Take the "read" path and allow the get method to determine + // which value to return, respectively either: + // + // 1. The entire cache object + // 2. The data stored at the key + // + if ( key === undefined || + ( ( key && typeof key === "string" ) && value === undefined ) ) { + + return this.get( owner, key ); + } + + // When the key is not a string, or both a key and value + // are specified, set or extend (existing objects) with either: + // + // 1. An object of properties + // 2. A key and value + // + this.set( owner, key, value ); + + // Since the "set" path can have two possible entry points + // return the expected data based on which path was taken[*] + return value !== undefined ? value : key; + }, + remove: function( owner, key ) { + var i, + cache = owner[ this.expando ]; + + if ( cache === undefined ) { + return; + } + + if ( key !== undefined ) { + + // Support array or space separated string of keys + if ( Array.isArray( key ) ) { + + // If key is an array of keys... + // We always set camelCase keys, so remove that. + key = key.map( camelCase ); + } else { + key = camelCase( key ); + + // If a key with the spaces exists, use it. + // Otherwise, create an array by matching non-whitespace + key = key in cache ? + [ key ] : + ( key.match( rnothtmlwhite ) || [] ); + } + + i = key.length; + + while ( i-- ) { + delete cache[ key[ i ] ]; + } + } + + // Remove the expando if there's no more data + if ( key === undefined || jQuery.isEmptyObject( cache ) ) { + + // Support: Chrome <=35 - 45 + // Webkit & Blink performance suffers when deleting properties + // from DOM nodes, so set to undefined instead + // https://bugs.chromium.org/p/chromium/issues/detail?id=378607 (bug restricted) + if ( owner.nodeType ) { + owner[ this.expando ] = undefined; + } else { + delete owner[ this.expando ]; + } + } + }, + hasData: function( owner ) { + var cache = owner[ this.expando ]; + return cache !== undefined && !jQuery.isEmptyObject( cache ); + } +}; +var dataPriv = new Data(); + +var dataUser = new Data(); + + + +// Implementation Summary +// +// 1. Enforce API surface and semantic compatibility with 1.9.x branch +// 2. Improve the module's maintainability by reducing the storage +// paths to a single mechanism. +// 3. Use the same single mechanism to support "private" and "user" data. +// 4. _Never_ expose "private" data to user code (TODO: Drop _data, _removeData) +// 5. Avoid exposing implementation details on user objects (eg. expando properties) +// 6. Provide a clear path for implementation upgrade to WeakMap in 2014 + +var rbrace = /^(?:\{[\w\W]*\}|\[[\w\W]*\])$/, + rmultiDash = /[A-Z]/g; + +function getData( data ) { + if ( data === "true" ) { + return true; + } + + if ( data === "false" ) { + return false; + } + + if ( data === "null" ) { + return null; + } + + // Only convert to a number if it doesn't change the string + if ( data === +data + "" ) { + return +data; + } + + if ( rbrace.test( data ) ) { + return JSON.parse( data ); + } + + return data; +} + +function dataAttr( elem, key, data ) { + var name; + + // If nothing was found internally, try to fetch any + // data from the HTML5 data-* attribute + if ( data === undefined && elem.nodeType === 1 ) { + name = "data-" + key.replace( rmultiDash, "-$&" ).toLowerCase(); + data = elem.getAttribute( name ); + + if ( typeof data === "string" ) { + try { + data = getData( data ); + } catch ( e ) {} + + // Make sure we set the data so it isn't changed later + dataUser.set( elem, key, data ); + } else { + data = undefined; + } + } + return data; +} + +jQuery.extend( { + hasData: function( elem ) { + return dataUser.hasData( elem ) || dataPriv.hasData( elem ); + }, + + data: function( elem, name, data ) { + return dataUser.access( elem, name, data ); + }, + + removeData: function( elem, name ) { + dataUser.remove( elem, name ); + }, + + // TODO: Now that all calls to _data and _removeData have been replaced + // with direct calls to dataPriv methods, these can be deprecated. + _data: function( elem, name, data ) { + return dataPriv.access( elem, name, data ); + }, + + _removeData: function( elem, name ) { + dataPriv.remove( elem, name ); + } +} ); + +jQuery.fn.extend( { + data: function( key, value ) { + var i, name, data, + elem = this[ 0 ], + attrs = elem && elem.attributes; + + // Gets all values + if ( key === undefined ) { + if ( this.length ) { + data = dataUser.get( elem ); + + if ( elem.nodeType === 1 && !dataPriv.get( elem, "hasDataAttrs" ) ) { + i = attrs.length; + while ( i-- ) { + + // Support: IE 11 only + // The attrs elements can be null (#14894) + if ( attrs[ i ] ) { + name = attrs[ i ].name; + if ( name.indexOf( "data-" ) === 0 ) { + name = camelCase( name.slice( 5 ) ); + dataAttr( elem, name, data[ name ] ); + } + } + } + dataPriv.set( elem, "hasDataAttrs", true ); + } + } + + return data; + } + + // Sets multiple values + if ( typeof key === "object" ) { + return this.each( function() { + dataUser.set( this, key ); + } ); + } + + return access( this, function( value ) { + var data; + + // The calling jQuery object (element matches) is not empty + // (and therefore has an element appears at this[ 0 ]) and the + // `value` parameter was not undefined. An empty jQuery object + // will result in `undefined` for elem = this[ 0 ] which will + // throw an exception if an attempt to read a data cache is made. + if ( elem && value === undefined ) { + + // Attempt to get data from the cache + // The key will always be camelCased in Data + data = dataUser.get( elem, key ); + if ( data !== undefined ) { + return data; + } + + // Attempt to "discover" the data in + // HTML5 custom data-* attrs + data = dataAttr( elem, key ); + if ( data !== undefined ) { + return data; + } + + // We tried really hard, but the data doesn't exist. + return; + } + + // Set the data... + this.each( function() { + + // We always store the camelCased key + dataUser.set( this, key, value ); + } ); + }, null, value, arguments.length > 1, null, true ); + }, + + removeData: function( key ) { + return this.each( function() { + dataUser.remove( this, key ); + } ); + } +} ); + + +jQuery.extend( { + queue: function( elem, type, data ) { + var queue; + + if ( elem ) { + type = ( type || "fx" ) + "queue"; + queue = dataPriv.get( elem, type ); + + // Speed up dequeue by getting out quickly if this is just a lookup + if ( data ) { + if ( !queue || Array.isArray( data ) ) { + queue = dataPriv.access( elem, type, jQuery.makeArray( data ) ); + } else { + queue.push( data ); + } + } + return queue || []; + } + }, + + dequeue: function( elem, type ) { + type = type || "fx"; + + var queue = jQuery.queue( elem, type ), + startLength = queue.length, + fn = queue.shift(), + hooks = jQuery._queueHooks( elem, type ), + next = function() { + jQuery.dequeue( elem, type ); + }; + + // If the fx queue is dequeued, always remove the progress sentinel + if ( fn === "inprogress" ) { + fn = queue.shift(); + startLength--; + } + + if ( fn ) { + + // Add a progress sentinel to prevent the fx queue from being + // automatically dequeued + if ( type === "fx" ) { + queue.unshift( "inprogress" ); + } + + // Clear up the last queue stop function + delete hooks.stop; + fn.call( elem, next, hooks ); + } + + if ( !startLength && hooks ) { + hooks.empty.fire(); + } + }, + + // Not public - generate a queueHooks object, or return the current one + _queueHooks: function( elem, type ) { + var key = type + "queueHooks"; + return dataPriv.get( elem, key ) || dataPriv.access( elem, key, { + empty: jQuery.Callbacks( "once memory" ).add( function() { + dataPriv.remove( elem, [ type + "queue", key ] ); + } ) + } ); + } +} ); + +jQuery.fn.extend( { + queue: function( type, data ) { + var setter = 2; + + if ( typeof type !== "string" ) { + data = type; + type = "fx"; + setter--; + } + + if ( arguments.length < setter ) { + return jQuery.queue( this[ 0 ], type ); + } + + return data === undefined ? + this : + this.each( function() { + var queue = jQuery.queue( this, type, data ); + + // Ensure a hooks for this queue + jQuery._queueHooks( this, type ); + + if ( type === "fx" && queue[ 0 ] !== "inprogress" ) { + jQuery.dequeue( this, type ); + } + } ); + }, + dequeue: function( type ) { + return this.each( function() { + jQuery.dequeue( this, type ); + } ); + }, + clearQueue: function( type ) { + return this.queue( type || "fx", [] ); + }, + + // Get a promise resolved when queues of a certain type + // are emptied (fx is the type by default) + promise: function( type, obj ) { + var tmp, + count = 1, + defer = jQuery.Deferred(), + elements = this, + i = this.length, + resolve = function() { + if ( !( --count ) ) { + defer.resolveWith( elements, [ elements ] ); + } + }; + + if ( typeof type !== "string" ) { + obj = type; + type = undefined; + } + type = type || "fx"; + + while ( i-- ) { + tmp = dataPriv.get( elements[ i ], type + "queueHooks" ); + if ( tmp && tmp.empty ) { + count++; + tmp.empty.add( resolve ); + } + } + resolve(); + return defer.promise( obj ); + } +} ); +var pnum = ( /[+-]?(?:\d*\.|)\d+(?:[eE][+-]?\d+|)/ ).source; + +var rcssNum = new RegExp( "^(?:([+-])=|)(" + pnum + ")([a-z%]*)$", "i" ); + + +var cssExpand = [ "Top", "Right", "Bottom", "Left" ]; + +var documentElement = document.documentElement; + + + + var isAttached = function( elem ) { + return jQuery.contains( elem.ownerDocument, elem ); + }, + composed = { composed: true }; + + // Support: IE 9 - 11+, Edge 12 - 18+, iOS 10.0 - 10.2 only + // Check attachment across shadow DOM boundaries when possible (gh-3504) + // Support: iOS 10.0-10.2 only + // Early iOS 10 versions support `attachShadow` but not `getRootNode`, + // leading to errors. We need to check for `getRootNode`. + if ( documentElement.getRootNode ) { + isAttached = function( elem ) { + return jQuery.contains( elem.ownerDocument, elem ) || + elem.getRootNode( composed ) === elem.ownerDocument; + }; + } +var isHiddenWithinTree = function( elem, el ) { + + // isHiddenWithinTree might be called from jQuery#filter function; + // in that case, element will be second argument + elem = el || elem; + + // Inline style trumps all + return elem.style.display === "none" || + elem.style.display === "" && + + // Otherwise, check computed style + // Support: Firefox <=43 - 45 + // Disconnected elements can have computed display: none, so first confirm that elem is + // in the document. + isAttached( elem ) && + + jQuery.css( elem, "display" ) === "none"; + }; + + + +function adjustCSS( elem, prop, valueParts, tween ) { + var adjusted, scale, + maxIterations = 20, + currentValue = tween ? + function() { + return tween.cur(); + } : + function() { + return jQuery.css( elem, prop, "" ); + }, + initial = currentValue(), + unit = valueParts && valueParts[ 3 ] || ( jQuery.cssNumber[ prop ] ? "" : "px" ), + + // Starting value computation is required for potential unit mismatches + initialInUnit = elem.nodeType && + ( jQuery.cssNumber[ prop ] || unit !== "px" && +initial ) && + rcssNum.exec( jQuery.css( elem, prop ) ); + + if ( initialInUnit && initialInUnit[ 3 ] !== unit ) { + + // Support: Firefox <=54 + // Halve the iteration target value to prevent interference from CSS upper bounds (gh-2144) + initial = initial / 2; + + // Trust units reported by jQuery.css + unit = unit || initialInUnit[ 3 ]; + + // Iteratively approximate from a nonzero starting point + initialInUnit = +initial || 1; + + while ( maxIterations-- ) { + + // Evaluate and update our best guess (doubling guesses that zero out). + // Finish if the scale equals or crosses 1 (making the old*new product non-positive). + jQuery.style( elem, prop, initialInUnit + unit ); + if ( ( 1 - scale ) * ( 1 - ( scale = currentValue() / initial || 0.5 ) ) <= 0 ) { + maxIterations = 0; + } + initialInUnit = initialInUnit / scale; + + } + + initialInUnit = initialInUnit * 2; + jQuery.style( elem, prop, initialInUnit + unit ); + + // Make sure we update the tween properties later on + valueParts = valueParts || []; + } + + if ( valueParts ) { + initialInUnit = +initialInUnit || +initial || 0; + + // Apply relative offset (+=/-=) if specified + adjusted = valueParts[ 1 ] ? + initialInUnit + ( valueParts[ 1 ] + 1 ) * valueParts[ 2 ] : + +valueParts[ 2 ]; + if ( tween ) { + tween.unit = unit; + tween.start = initialInUnit; + tween.end = adjusted; + } + } + return adjusted; +} + + +var defaultDisplayMap = {}; + +function getDefaultDisplay( elem ) { + var temp, + doc = elem.ownerDocument, + nodeName = elem.nodeName, + display = defaultDisplayMap[ nodeName ]; + + if ( display ) { + return display; + } + + temp = doc.body.appendChild( doc.createElement( nodeName ) ); + display = jQuery.css( temp, "display" ); + + temp.parentNode.removeChild( temp ); + + if ( display === "none" ) { + display = "block"; + } + defaultDisplayMap[ nodeName ] = display; + + return display; +} + +function showHide( elements, show ) { + var display, elem, + values = [], + index = 0, + length = elements.length; + + // Determine new display value for elements that need to change + for ( ; index < length; index++ ) { + elem = elements[ index ]; + if ( !elem.style ) { + continue; + } + + display = elem.style.display; + if ( show ) { + + // Since we force visibility upon cascade-hidden elements, an immediate (and slow) + // check is required in this first loop unless we have a nonempty display value (either + // inline or about-to-be-restored) + if ( display === "none" ) { + values[ index ] = dataPriv.get( elem, "display" ) || null; + if ( !values[ index ] ) { + elem.style.display = ""; + } + } + if ( elem.style.display === "" && isHiddenWithinTree( elem ) ) { + values[ index ] = getDefaultDisplay( elem ); + } + } else { + if ( display !== "none" ) { + values[ index ] = "none"; + + // Remember what we're overwriting + dataPriv.set( elem, "display", display ); + } + } + } + + // Set the display of the elements in a second loop to avoid constant reflow + for ( index = 0; index < length; index++ ) { + if ( values[ index ] != null ) { + elements[ index ].style.display = values[ index ]; + } + } + + return elements; +} + +jQuery.fn.extend( { + show: function() { + return showHide( this, true ); + }, + hide: function() { + return showHide( this ); + }, + toggle: function( state ) { + if ( typeof state === "boolean" ) { + return state ? this.show() : this.hide(); + } + + return this.each( function() { + if ( isHiddenWithinTree( this ) ) { + jQuery( this ).show(); + } else { + jQuery( this ).hide(); + } + } ); + } +} ); +var rcheckableType = ( /^(?:checkbox|radio)$/i ); + +var rtagName = ( /<([a-z][^\/\0>\x20\t\r\n\f]*)/i ); + +var rscriptType = ( /^$|^module$|\/(?:java|ecma)script/i ); + + + +( function() { + var fragment = document.createDocumentFragment(), + div = fragment.appendChild( document.createElement( "div" ) ), + input = document.createElement( "input" ); + + // Support: Android 4.0 - 4.3 only + // Check state lost if the name is set (#11217) + // Support: Windows Web Apps (WWA) + // `name` and `type` must use .setAttribute for WWA (#14901) + input.setAttribute( "type", "radio" ); + input.setAttribute( "checked", "checked" ); + input.setAttribute( "name", "t" ); + + div.appendChild( input ); + + // Support: Android <=4.1 only + // Older WebKit doesn't clone checked state correctly in fragments + support.checkClone = div.cloneNode( true ).cloneNode( true ).lastChild.checked; + + // Support: IE <=11 only + // Make sure textarea (and checkbox) defaultValue is properly cloned + div.innerHTML = ""; + support.noCloneChecked = !!div.cloneNode( true ).lastChild.defaultValue; + + // Support: IE <=9 only + // IE <=9 replaces "; + support.option = !!div.lastChild; +} )(); + + +// We have to close these tags to support XHTML (#13200) +var wrapMap = { + + // XHTML parsers do not magically insert elements in the + // same way that tag soup parsers do. So we cannot shorten + // this by omitting or other required elements. + thead: [ 1, "", "
" ], + col: [ 2, "", "
" ], + tr: [ 2, "", "
" ], + td: [ 3, "", "
" ], + + _default: [ 0, "", "" ] +}; + +wrapMap.tbody = wrapMap.tfoot = wrapMap.colgroup = wrapMap.caption = wrapMap.thead; +wrapMap.th = wrapMap.td; + +// Support: IE <=9 only +if ( !support.option ) { + wrapMap.optgroup = wrapMap.option = [ 1, "" ]; +} + + +function getAll( context, tag ) { + + // Support: IE <=9 - 11 only + // Use typeof to avoid zero-argument method invocation on host objects (#15151) + var ret; + + if ( typeof context.getElementsByTagName !== "undefined" ) { + ret = context.getElementsByTagName( tag || "*" ); + + } else if ( typeof context.querySelectorAll !== "undefined" ) { + ret = context.querySelectorAll( tag || "*" ); + + } else { + ret = []; + } + + if ( tag === undefined || tag && nodeName( context, tag ) ) { + return jQuery.merge( [ context ], ret ); + } + + return ret; +} + + +// Mark scripts as having already been evaluated +function setGlobalEval( elems, refElements ) { + var i = 0, + l = elems.length; + + for ( ; i < l; i++ ) { + dataPriv.set( + elems[ i ], + "globalEval", + !refElements || dataPriv.get( refElements[ i ], "globalEval" ) + ); + } +} + + +var rhtml = /<|&#?\w+;/; + +function buildFragment( elems, context, scripts, selection, ignored ) { + var elem, tmp, tag, wrap, attached, j, + fragment = context.createDocumentFragment(), + nodes = [], + i = 0, + l = elems.length; + + for ( ; i < l; i++ ) { + elem = elems[ i ]; + + if ( elem || elem === 0 ) { + + // Add nodes directly + if ( toType( elem ) === "object" ) { + + // Support: Android <=4.0 only, PhantomJS 1 only + // push.apply(_, arraylike) throws on ancient WebKit + jQuery.merge( nodes, elem.nodeType ? [ elem ] : elem ); + + // Convert non-html into a text node + } else if ( !rhtml.test( elem ) ) { + nodes.push( context.createTextNode( elem ) ); + + // Convert html into DOM nodes + } else { + tmp = tmp || fragment.appendChild( context.createElement( "div" ) ); + + // Deserialize a standard representation + tag = ( rtagName.exec( elem ) || [ "", "" ] )[ 1 ].toLowerCase(); + wrap = wrapMap[ tag ] || wrapMap._default; + tmp.innerHTML = wrap[ 1 ] + jQuery.htmlPrefilter( elem ) + wrap[ 2 ]; + + // Descend through wrappers to the right content + j = wrap[ 0 ]; + while ( j-- ) { + tmp = tmp.lastChild; + } + + // Support: Android <=4.0 only, PhantomJS 1 only + // push.apply(_, arraylike) throws on ancient WebKit + jQuery.merge( nodes, tmp.childNodes ); + + // Remember the top-level container + tmp = fragment.firstChild; + + // Ensure the created nodes are orphaned (#12392) + tmp.textContent = ""; + } + } + } + + // Remove wrapper from fragment + fragment.textContent = ""; + + i = 0; + while ( ( elem = nodes[ i++ ] ) ) { + + // Skip elements already in the context collection (trac-4087) + if ( selection && jQuery.inArray( elem, selection ) > -1 ) { + if ( ignored ) { + ignored.push( elem ); + } + continue; + } + + attached = isAttached( elem ); + + // Append to fragment + tmp = getAll( fragment.appendChild( elem ), "script" ); + + // Preserve script evaluation history + if ( attached ) { + setGlobalEval( tmp ); + } + + // Capture executables + if ( scripts ) { + j = 0; + while ( ( elem = tmp[ j++ ] ) ) { + if ( rscriptType.test( elem.type || "" ) ) { + scripts.push( elem ); + } + } + } + } + + return fragment; +} + + +var rtypenamespace = /^([^.]*)(?:\.(.+)|)/; + +function returnTrue() { + return true; +} + +function returnFalse() { + return false; +} + +// Support: IE <=9 - 11+ +// focus() and blur() are asynchronous, except when they are no-op. +// So expect focus to be synchronous when the element is already active, +// and blur to be synchronous when the element is not already active. +// (focus and blur are always synchronous in other supported browsers, +// this just defines when we can count on it). +function expectSync( elem, type ) { + return ( elem === safeActiveElement() ) === ( type === "focus" ); +} + +// Support: IE <=9 only +// Accessing document.activeElement can throw unexpectedly +// https://bugs.jquery.com/ticket/13393 +function safeActiveElement() { + try { + return document.activeElement; + } catch ( err ) { } +} + +function on( elem, types, selector, data, fn, one ) { + var origFn, type; + + // Types can be a map of types/handlers + if ( typeof types === "object" ) { + + // ( types-Object, selector, data ) + if ( typeof selector !== "string" ) { + + // ( types-Object, data ) + data = data || selector; + selector = undefined; + } + for ( type in types ) { + on( elem, type, selector, data, types[ type ], one ); + } + return elem; + } + + if ( data == null && fn == null ) { + + // ( types, fn ) + fn = selector; + data = selector = undefined; + } else if ( fn == null ) { + if ( typeof selector === "string" ) { + + // ( types, selector, fn ) + fn = data; + data = undefined; + } else { + + // ( types, data, fn ) + fn = data; + data = selector; + selector = undefined; + } + } + if ( fn === false ) { + fn = returnFalse; + } else if ( !fn ) { + return elem; + } + + if ( one === 1 ) { + origFn = fn; + fn = function( event ) { + + // Can use an empty set, since event contains the info + jQuery().off( event ); + return origFn.apply( this, arguments ); + }; + + // Use same guid so caller can remove using origFn + fn.guid = origFn.guid || ( origFn.guid = jQuery.guid++ ); + } + return elem.each( function() { + jQuery.event.add( this, types, fn, data, selector ); + } ); +} + +/* + * Helper functions for managing events -- not part of the public interface. + * Props to Dean Edwards' addEvent library for many of the ideas. + */ +jQuery.event = { + + global: {}, + + add: function( elem, types, handler, data, selector ) { + + var handleObjIn, eventHandle, tmp, + events, t, handleObj, + special, handlers, type, namespaces, origType, + elemData = dataPriv.get( elem ); + + // Only attach events to objects that accept data + if ( !acceptData( elem ) ) { + return; + } + + // Caller can pass in an object of custom data in lieu of the handler + if ( handler.handler ) { + handleObjIn = handler; + handler = handleObjIn.handler; + selector = handleObjIn.selector; + } + + // Ensure that invalid selectors throw exceptions at attach time + // Evaluate against documentElement in case elem is a non-element node (e.g., document) + if ( selector ) { + jQuery.find.matchesSelector( documentElement, selector ); + } + + // Make sure that the handler has a unique ID, used to find/remove it later + if ( !handler.guid ) { + handler.guid = jQuery.guid++; + } + + // Init the element's event structure and main handler, if this is the first + if ( !( events = elemData.events ) ) { + events = elemData.events = Object.create( null ); + } + if ( !( eventHandle = elemData.handle ) ) { + eventHandle = elemData.handle = function( e ) { + + // Discard the second event of a jQuery.event.trigger() and + // when an event is called after a page has unloaded + return typeof jQuery !== "undefined" && jQuery.event.triggered !== e.type ? + jQuery.event.dispatch.apply( elem, arguments ) : undefined; + }; + } + + // Handle multiple events separated by a space + types = ( types || "" ).match( rnothtmlwhite ) || [ "" ]; + t = types.length; + while ( t-- ) { + tmp = rtypenamespace.exec( types[ t ] ) || []; + type = origType = tmp[ 1 ]; + namespaces = ( tmp[ 2 ] || "" ).split( "." ).sort(); + + // There *must* be a type, no attaching namespace-only handlers + if ( !type ) { + continue; + } + + // If event changes its type, use the special event handlers for the changed type + special = jQuery.event.special[ type ] || {}; + + // If selector defined, determine special event api type, otherwise given type + type = ( selector ? special.delegateType : special.bindType ) || type; + + // Update special based on newly reset type + special = jQuery.event.special[ type ] || {}; + + // handleObj is passed to all event handlers + handleObj = jQuery.extend( { + type: type, + origType: origType, + data: data, + handler: handler, + guid: handler.guid, + selector: selector, + needsContext: selector && jQuery.expr.match.needsContext.test( selector ), + namespace: namespaces.join( "." ) + }, handleObjIn ); + + // Init the event handler queue if we're the first + if ( !( handlers = events[ type ] ) ) { + handlers = events[ type ] = []; + handlers.delegateCount = 0; + + // Only use addEventListener if the special events handler returns false + if ( !special.setup || + special.setup.call( elem, data, namespaces, eventHandle ) === false ) { + + if ( elem.addEventListener ) { + elem.addEventListener( type, eventHandle ); + } + } + } + + if ( special.add ) { + special.add.call( elem, handleObj ); + + if ( !handleObj.handler.guid ) { + handleObj.handler.guid = handler.guid; + } + } + + // Add to the element's handler list, delegates in front + if ( selector ) { + handlers.splice( handlers.delegateCount++, 0, handleObj ); + } else { + handlers.push( handleObj ); + } + + // Keep track of which events have ever been used, for event optimization + jQuery.event.global[ type ] = true; + } + + }, + + // Detach an event or set of events from an element + remove: function( elem, types, handler, selector, mappedTypes ) { + + var j, origCount, tmp, + events, t, handleObj, + special, handlers, type, namespaces, origType, + elemData = dataPriv.hasData( elem ) && dataPriv.get( elem ); + + if ( !elemData || !( events = elemData.events ) ) { + return; + } + + // Once for each type.namespace in types; type may be omitted + types = ( types || "" ).match( rnothtmlwhite ) || [ "" ]; + t = types.length; + while ( t-- ) { + tmp = rtypenamespace.exec( types[ t ] ) || []; + type = origType = tmp[ 1 ]; + namespaces = ( tmp[ 2 ] || "" ).split( "." ).sort(); + + // Unbind all events (on this namespace, if provided) for the element + if ( !type ) { + for ( type in events ) { + jQuery.event.remove( elem, type + types[ t ], handler, selector, true ); + } + continue; + } + + special = jQuery.event.special[ type ] || {}; + type = ( selector ? special.delegateType : special.bindType ) || type; + handlers = events[ type ] || []; + tmp = tmp[ 2 ] && + new RegExp( "(^|\\.)" + namespaces.join( "\\.(?:.*\\.|)" ) + "(\\.|$)" ); + + // Remove matching events + origCount = j = handlers.length; + while ( j-- ) { + handleObj = handlers[ j ]; + + if ( ( mappedTypes || origType === handleObj.origType ) && + ( !handler || handler.guid === handleObj.guid ) && + ( !tmp || tmp.test( handleObj.namespace ) ) && + ( !selector || selector === handleObj.selector || + selector === "**" && handleObj.selector ) ) { + handlers.splice( j, 1 ); + + if ( handleObj.selector ) { + handlers.delegateCount--; + } + if ( special.remove ) { + special.remove.call( elem, handleObj ); + } + } + } + + // Remove generic event handler if we removed something and no more handlers exist + // (avoids potential for endless recursion during removal of special event handlers) + if ( origCount && !handlers.length ) { + if ( !special.teardown || + special.teardown.call( elem, namespaces, elemData.handle ) === false ) { + + jQuery.removeEvent( elem, type, elemData.handle ); + } + + delete events[ type ]; + } + } + + // Remove data and the expando if it's no longer used + if ( jQuery.isEmptyObject( events ) ) { + dataPriv.remove( elem, "handle events" ); + } + }, + + dispatch: function( nativeEvent ) { + + var i, j, ret, matched, handleObj, handlerQueue, + args = new Array( arguments.length ), + + // Make a writable jQuery.Event from the native event object + event = jQuery.event.fix( nativeEvent ), + + handlers = ( + dataPriv.get( this, "events" ) || Object.create( null ) + )[ event.type ] || [], + special = jQuery.event.special[ event.type ] || {}; + + // Use the fix-ed jQuery.Event rather than the (read-only) native event + args[ 0 ] = event; + + for ( i = 1; i < arguments.length; i++ ) { + args[ i ] = arguments[ i ]; + } + + event.delegateTarget = this; + + // Call the preDispatch hook for the mapped type, and let it bail if desired + if ( special.preDispatch && special.preDispatch.call( this, event ) === false ) { + return; + } + + // Determine handlers + handlerQueue = jQuery.event.handlers.call( this, event, handlers ); + + // Run delegates first; they may want to stop propagation beneath us + i = 0; + while ( ( matched = handlerQueue[ i++ ] ) && !event.isPropagationStopped() ) { + event.currentTarget = matched.elem; + + j = 0; + while ( ( handleObj = matched.handlers[ j++ ] ) && + !event.isImmediatePropagationStopped() ) { + + // If the event is namespaced, then each handler is only invoked if it is + // specially universal or its namespaces are a superset of the event's. + if ( !event.rnamespace || handleObj.namespace === false || + event.rnamespace.test( handleObj.namespace ) ) { + + event.handleObj = handleObj; + event.data = handleObj.data; + + ret = ( ( jQuery.event.special[ handleObj.origType ] || {} ).handle || + handleObj.handler ).apply( matched.elem, args ); + + if ( ret !== undefined ) { + if ( ( event.result = ret ) === false ) { + event.preventDefault(); + event.stopPropagation(); + } + } + } + } + } + + // Call the postDispatch hook for the mapped type + if ( special.postDispatch ) { + special.postDispatch.call( this, event ); + } + + return event.result; + }, + + handlers: function( event, handlers ) { + var i, handleObj, sel, matchedHandlers, matchedSelectors, + handlerQueue = [], + delegateCount = handlers.delegateCount, + cur = event.target; + + // Find delegate handlers + if ( delegateCount && + + // Support: IE <=9 + // Black-hole SVG instance trees (trac-13180) + cur.nodeType && + + // Support: Firefox <=42 + // Suppress spec-violating clicks indicating a non-primary pointer button (trac-3861) + // https://www.w3.org/TR/DOM-Level-3-Events/#event-type-click + // Support: IE 11 only + // ...but not arrow key "clicks" of radio inputs, which can have `button` -1 (gh-2343) + !( event.type === "click" && event.button >= 1 ) ) { + + for ( ; cur !== this; cur = cur.parentNode || this ) { + + // Don't check non-elements (#13208) + // Don't process clicks on disabled elements (#6911, #8165, #11382, #11764) + if ( cur.nodeType === 1 && !( event.type === "click" && cur.disabled === true ) ) { + matchedHandlers = []; + matchedSelectors = {}; + for ( i = 0; i < delegateCount; i++ ) { + handleObj = handlers[ i ]; + + // Don't conflict with Object.prototype properties (#13203) + sel = handleObj.selector + " "; + + if ( matchedSelectors[ sel ] === undefined ) { + matchedSelectors[ sel ] = handleObj.needsContext ? + jQuery( sel, this ).index( cur ) > -1 : + jQuery.find( sel, this, null, [ cur ] ).length; + } + if ( matchedSelectors[ sel ] ) { + matchedHandlers.push( handleObj ); + } + } + if ( matchedHandlers.length ) { + handlerQueue.push( { elem: cur, handlers: matchedHandlers } ); + } + } + } + } + + // Add the remaining (directly-bound) handlers + cur = this; + if ( delegateCount < handlers.length ) { + handlerQueue.push( { elem: cur, handlers: handlers.slice( delegateCount ) } ); + } + + return handlerQueue; + }, + + addProp: function( name, hook ) { + Object.defineProperty( jQuery.Event.prototype, name, { + enumerable: true, + configurable: true, + + get: isFunction( hook ) ? + function() { + if ( this.originalEvent ) { + return hook( this.originalEvent ); + } + } : + function() { + if ( this.originalEvent ) { + return this.originalEvent[ name ]; + } + }, + + set: function( value ) { + Object.defineProperty( this, name, { + enumerable: true, + configurable: true, + writable: true, + value: value + } ); + } + } ); + }, + + fix: function( originalEvent ) { + return originalEvent[ jQuery.expando ] ? + originalEvent : + new jQuery.Event( originalEvent ); + }, + + special: { + load: { + + // Prevent triggered image.load events from bubbling to window.load + noBubble: true + }, + click: { + + // Utilize native event to ensure correct state for checkable inputs + setup: function( data ) { + + // For mutual compressibility with _default, replace `this` access with a local var. + // `|| data` is dead code meant only to preserve the variable through minification. + var el = this || data; + + // Claim the first handler + if ( rcheckableType.test( el.type ) && + el.click && nodeName( el, "input" ) ) { + + // dataPriv.set( el, "click", ... ) + leverageNative( el, "click", returnTrue ); + } + + // Return false to allow normal processing in the caller + return false; + }, + trigger: function( data ) { + + // For mutual compressibility with _default, replace `this` access with a local var. + // `|| data` is dead code meant only to preserve the variable through minification. + var el = this || data; + + // Force setup before triggering a click + if ( rcheckableType.test( el.type ) && + el.click && nodeName( el, "input" ) ) { + + leverageNative( el, "click" ); + } + + // Return non-false to allow normal event-path propagation + return true; + }, + + // For cross-browser consistency, suppress native .click() on links + // Also prevent it if we're currently inside a leveraged native-event stack + _default: function( event ) { + var target = event.target; + return rcheckableType.test( target.type ) && + target.click && nodeName( target, "input" ) && + dataPriv.get( target, "click" ) || + nodeName( target, "a" ); + } + }, + + beforeunload: { + postDispatch: function( event ) { + + // Support: Firefox 20+ + // Firefox doesn't alert if the returnValue field is not set. + if ( event.result !== undefined && event.originalEvent ) { + event.originalEvent.returnValue = event.result; + } + } + } + } +}; + +// Ensure the presence of an event listener that handles manually-triggered +// synthetic events by interrupting progress until reinvoked in response to +// *native* events that it fires directly, ensuring that state changes have +// already occurred before other listeners are invoked. +function leverageNative( el, type, expectSync ) { + + // Missing expectSync indicates a trigger call, which must force setup through jQuery.event.add + if ( !expectSync ) { + if ( dataPriv.get( el, type ) === undefined ) { + jQuery.event.add( el, type, returnTrue ); + } + return; + } + + // Register the controller as a special universal handler for all event namespaces + dataPriv.set( el, type, false ); + jQuery.event.add( el, type, { + namespace: false, + handler: function( event ) { + var notAsync, result, + saved = dataPriv.get( this, type ); + + if ( ( event.isTrigger & 1 ) && this[ type ] ) { + + // Interrupt processing of the outer synthetic .trigger()ed event + // Saved data should be false in such cases, but might be a leftover capture object + // from an async native handler (gh-4350) + if ( !saved.length ) { + + // Store arguments for use when handling the inner native event + // There will always be at least one argument (an event object), so this array + // will not be confused with a leftover capture object. + saved = slice.call( arguments ); + dataPriv.set( this, type, saved ); + + // Trigger the native event and capture its result + // Support: IE <=9 - 11+ + // focus() and blur() are asynchronous + notAsync = expectSync( this, type ); + this[ type ](); + result = dataPriv.get( this, type ); + if ( saved !== result || notAsync ) { + dataPriv.set( this, type, false ); + } else { + result = {}; + } + if ( saved !== result ) { + + // Cancel the outer synthetic event + event.stopImmediatePropagation(); + event.preventDefault(); + + // Support: Chrome 86+ + // In Chrome, if an element having a focusout handler is blurred by + // clicking outside of it, it invokes the handler synchronously. If + // that handler calls `.remove()` on the element, the data is cleared, + // leaving `result` undefined. We need to guard against this. + return result && result.value; + } + + // If this is an inner synthetic event for an event with a bubbling surrogate + // (focus or blur), assume that the surrogate already propagated from triggering the + // native event and prevent that from happening again here. + // This technically gets the ordering wrong w.r.t. to `.trigger()` (in which the + // bubbling surrogate propagates *after* the non-bubbling base), but that seems + // less bad than duplication. + } else if ( ( jQuery.event.special[ type ] || {} ).delegateType ) { + event.stopPropagation(); + } + + // If this is a native event triggered above, everything is now in order + // Fire an inner synthetic event with the original arguments + } else if ( saved.length ) { + + // ...and capture the result + dataPriv.set( this, type, { + value: jQuery.event.trigger( + + // Support: IE <=9 - 11+ + // Extend with the prototype to reset the above stopImmediatePropagation() + jQuery.extend( saved[ 0 ], jQuery.Event.prototype ), + saved.slice( 1 ), + this + ) + } ); + + // Abort handling of the native event + event.stopImmediatePropagation(); + } + } + } ); +} + +jQuery.removeEvent = function( elem, type, handle ) { + + // This "if" is needed for plain objects + if ( elem.removeEventListener ) { + elem.removeEventListener( type, handle ); + } +}; + +jQuery.Event = function( src, props ) { + + // Allow instantiation without the 'new' keyword + if ( !( this instanceof jQuery.Event ) ) { + return new jQuery.Event( src, props ); + } + + // Event object + if ( src && src.type ) { + this.originalEvent = src; + this.type = src.type; + + // Events bubbling up the document may have been marked as prevented + // by a handler lower down the tree; reflect the correct value. + this.isDefaultPrevented = src.defaultPrevented || + src.defaultPrevented === undefined && + + // Support: Android <=2.3 only + src.returnValue === false ? + returnTrue : + returnFalse; + + // Create target properties + // Support: Safari <=6 - 7 only + // Target should not be a text node (#504, #13143) + this.target = ( src.target && src.target.nodeType === 3 ) ? + src.target.parentNode : + src.target; + + this.currentTarget = src.currentTarget; + this.relatedTarget = src.relatedTarget; + + // Event type + } else { + this.type = src; + } + + // Put explicitly provided properties onto the event object + if ( props ) { + jQuery.extend( this, props ); + } + + // Create a timestamp if incoming event doesn't have one + this.timeStamp = src && src.timeStamp || Date.now(); + + // Mark it as fixed + this[ jQuery.expando ] = true; +}; + +// jQuery.Event is based on DOM3 Events as specified by the ECMAScript Language Binding +// https://www.w3.org/TR/2003/WD-DOM-Level-3-Events-20030331/ecma-script-binding.html +jQuery.Event.prototype = { + constructor: jQuery.Event, + isDefaultPrevented: returnFalse, + isPropagationStopped: returnFalse, + isImmediatePropagationStopped: returnFalse, + isSimulated: false, + + preventDefault: function() { + var e = this.originalEvent; + + this.isDefaultPrevented = returnTrue; + + if ( e && !this.isSimulated ) { + e.preventDefault(); + } + }, + stopPropagation: function() { + var e = this.originalEvent; + + this.isPropagationStopped = returnTrue; + + if ( e && !this.isSimulated ) { + e.stopPropagation(); + } + }, + stopImmediatePropagation: function() { + var e = this.originalEvent; + + this.isImmediatePropagationStopped = returnTrue; + + if ( e && !this.isSimulated ) { + e.stopImmediatePropagation(); + } + + this.stopPropagation(); + } +}; + +// Includes all common event props including KeyEvent and MouseEvent specific props +jQuery.each( { + altKey: true, + bubbles: true, + cancelable: true, + changedTouches: true, + ctrlKey: true, + detail: true, + eventPhase: true, + metaKey: true, + pageX: true, + pageY: true, + shiftKey: true, + view: true, + "char": true, + code: true, + charCode: true, + key: true, + keyCode: true, + button: true, + buttons: true, + clientX: true, + clientY: true, + offsetX: true, + offsetY: true, + pointerId: true, + pointerType: true, + screenX: true, + screenY: true, + targetTouches: true, + toElement: true, + touches: true, + which: true +}, jQuery.event.addProp ); + +jQuery.each( { focus: "focusin", blur: "focusout" }, function( type, delegateType ) { + jQuery.event.special[ type ] = { + + // Utilize native event if possible so blur/focus sequence is correct + setup: function() { + + // Claim the first handler + // dataPriv.set( this, "focus", ... ) + // dataPriv.set( this, "blur", ... ) + leverageNative( this, type, expectSync ); + + // Return false to allow normal processing in the caller + return false; + }, + trigger: function() { + + // Force setup before trigger + leverageNative( this, type ); + + // Return non-false to allow normal event-path propagation + return true; + }, + + // Suppress native focus or blur as it's already being fired + // in leverageNative. + _default: function() { + return true; + }, + + delegateType: delegateType + }; +} ); + +// Create mouseenter/leave events using mouseover/out and event-time checks +// so that event delegation works in jQuery. +// Do the same for pointerenter/pointerleave and pointerover/pointerout +// +// Support: Safari 7 only +// Safari sends mouseenter too often; see: +// https://bugs.chromium.org/p/chromium/issues/detail?id=470258 +// for the description of the bug (it existed in older Chrome versions as well). +jQuery.each( { + mouseenter: "mouseover", + mouseleave: "mouseout", + pointerenter: "pointerover", + pointerleave: "pointerout" +}, function( orig, fix ) { + jQuery.event.special[ orig ] = { + delegateType: fix, + bindType: fix, + + handle: function( event ) { + var ret, + target = this, + related = event.relatedTarget, + handleObj = event.handleObj; + + // For mouseenter/leave call the handler if related is outside the target. + // NB: No relatedTarget if the mouse left/entered the browser window + if ( !related || ( related !== target && !jQuery.contains( target, related ) ) ) { + event.type = handleObj.origType; + ret = handleObj.handler.apply( this, arguments ); + event.type = fix; + } + return ret; + } + }; +} ); + +jQuery.fn.extend( { + + on: function( types, selector, data, fn ) { + return on( this, types, selector, data, fn ); + }, + one: function( types, selector, data, fn ) { + return on( this, types, selector, data, fn, 1 ); + }, + off: function( types, selector, fn ) { + var handleObj, type; + if ( types && types.preventDefault && types.handleObj ) { + + // ( event ) dispatched jQuery.Event + handleObj = types.handleObj; + jQuery( types.delegateTarget ).off( + handleObj.namespace ? + handleObj.origType + "." + handleObj.namespace : + handleObj.origType, + handleObj.selector, + handleObj.handler + ); + return this; + } + if ( typeof types === "object" ) { + + // ( types-object [, selector] ) + for ( type in types ) { + this.off( type, selector, types[ type ] ); + } + return this; + } + if ( selector === false || typeof selector === "function" ) { + + // ( types [, fn] ) + fn = selector; + selector = undefined; + } + if ( fn === false ) { + fn = returnFalse; + } + return this.each( function() { + jQuery.event.remove( this, types, fn, selector ); + } ); + } +} ); + + +var + + // Support: IE <=10 - 11, Edge 12 - 13 only + // In IE/Edge using regex groups here causes severe slowdowns. + // See https://connect.microsoft.com/IE/feedback/details/1736512/ + rnoInnerhtml = /\s*$/g; + +// Prefer a tbody over its parent table for containing new rows +function manipulationTarget( elem, content ) { + if ( nodeName( elem, "table" ) && + nodeName( content.nodeType !== 11 ? content : content.firstChild, "tr" ) ) { + + return jQuery( elem ).children( "tbody" )[ 0 ] || elem; + } + + return elem; +} + +// Replace/restore the type attribute of script elements for safe DOM manipulation +function disableScript( elem ) { + elem.type = ( elem.getAttribute( "type" ) !== null ) + "/" + elem.type; + return elem; +} +function restoreScript( elem ) { + if ( ( elem.type || "" ).slice( 0, 5 ) === "true/" ) { + elem.type = elem.type.slice( 5 ); + } else { + elem.removeAttribute( "type" ); + } + + return elem; +} + +function cloneCopyEvent( src, dest ) { + var i, l, type, pdataOld, udataOld, udataCur, events; + + if ( dest.nodeType !== 1 ) { + return; + } + + // 1. Copy private data: events, handlers, etc. + if ( dataPriv.hasData( src ) ) { + pdataOld = dataPriv.get( src ); + events = pdataOld.events; + + if ( events ) { + dataPriv.remove( dest, "handle events" ); + + for ( type in events ) { + for ( i = 0, l = events[ type ].length; i < l; i++ ) { + jQuery.event.add( dest, type, events[ type ][ i ] ); + } + } + } + } + + // 2. Copy user data + if ( dataUser.hasData( src ) ) { + udataOld = dataUser.access( src ); + udataCur = jQuery.extend( {}, udataOld ); + + dataUser.set( dest, udataCur ); + } +} + +// Fix IE bugs, see support tests +function fixInput( src, dest ) { + var nodeName = dest.nodeName.toLowerCase(); + + // Fails to persist the checked state of a cloned checkbox or radio button. + if ( nodeName === "input" && rcheckableType.test( src.type ) ) { + dest.checked = src.checked; + + // Fails to return the selected option to the default selected state when cloning options + } else if ( nodeName === "input" || nodeName === "textarea" ) { + dest.defaultValue = src.defaultValue; + } +} + +function domManip( collection, args, callback, ignored ) { + + // Flatten any nested arrays + args = flat( args ); + + var fragment, first, scripts, hasScripts, node, doc, + i = 0, + l = collection.length, + iNoClone = l - 1, + value = args[ 0 ], + valueIsFunction = isFunction( value ); + + // We can't cloneNode fragments that contain checked, in WebKit + if ( valueIsFunction || + ( l > 1 && typeof value === "string" && + !support.checkClone && rchecked.test( value ) ) ) { + return collection.each( function( index ) { + var self = collection.eq( index ); + if ( valueIsFunction ) { + args[ 0 ] = value.call( this, index, self.html() ); + } + domManip( self, args, callback, ignored ); + } ); + } + + if ( l ) { + fragment = buildFragment( args, collection[ 0 ].ownerDocument, false, collection, ignored ); + first = fragment.firstChild; + + if ( fragment.childNodes.length === 1 ) { + fragment = first; + } + + // Require either new content or an interest in ignored elements to invoke the callback + if ( first || ignored ) { + scripts = jQuery.map( getAll( fragment, "script" ), disableScript ); + hasScripts = scripts.length; + + // Use the original fragment for the last item + // instead of the first because it can end up + // being emptied incorrectly in certain situations (#8070). + for ( ; i < l; i++ ) { + node = fragment; + + if ( i !== iNoClone ) { + node = jQuery.clone( node, true, true ); + + // Keep references to cloned scripts for later restoration + if ( hasScripts ) { + + // Support: Android <=4.0 only, PhantomJS 1 only + // push.apply(_, arraylike) throws on ancient WebKit + jQuery.merge( scripts, getAll( node, "script" ) ); + } + } + + callback.call( collection[ i ], node, i ); + } + + if ( hasScripts ) { + doc = scripts[ scripts.length - 1 ].ownerDocument; + + // Reenable scripts + jQuery.map( scripts, restoreScript ); + + // Evaluate executable scripts on first document insertion + for ( i = 0; i < hasScripts; i++ ) { + node = scripts[ i ]; + if ( rscriptType.test( node.type || "" ) && + !dataPriv.access( node, "globalEval" ) && + jQuery.contains( doc, node ) ) { + + if ( node.src && ( node.type || "" ).toLowerCase() !== "module" ) { + + // Optional AJAX dependency, but won't run scripts if not present + if ( jQuery._evalUrl && !node.noModule ) { + jQuery._evalUrl( node.src, { + nonce: node.nonce || node.getAttribute( "nonce" ) + }, doc ); + } + } else { + DOMEval( node.textContent.replace( rcleanScript, "" ), node, doc ); + } + } + } + } + } + } + + return collection; +} + +function remove( elem, selector, keepData ) { + var node, + nodes = selector ? jQuery.filter( selector, elem ) : elem, + i = 0; + + for ( ; ( node = nodes[ i ] ) != null; i++ ) { + if ( !keepData && node.nodeType === 1 ) { + jQuery.cleanData( getAll( node ) ); + } + + if ( node.parentNode ) { + if ( keepData && isAttached( node ) ) { + setGlobalEval( getAll( node, "script" ) ); + } + node.parentNode.removeChild( node ); + } + } + + return elem; +} + +jQuery.extend( { + htmlPrefilter: function( html ) { + return html; + }, + + clone: function( elem, dataAndEvents, deepDataAndEvents ) { + var i, l, srcElements, destElements, + clone = elem.cloneNode( true ), + inPage = isAttached( elem ); + + // Fix IE cloning issues + if ( !support.noCloneChecked && ( elem.nodeType === 1 || elem.nodeType === 11 ) && + !jQuery.isXMLDoc( elem ) ) { + + // We eschew Sizzle here for performance reasons: https://jsperf.com/getall-vs-sizzle/2 + destElements = getAll( clone ); + srcElements = getAll( elem ); + + for ( i = 0, l = srcElements.length; i < l; i++ ) { + fixInput( srcElements[ i ], destElements[ i ] ); + } + } + + // Copy the events from the original to the clone + if ( dataAndEvents ) { + if ( deepDataAndEvents ) { + srcElements = srcElements || getAll( elem ); + destElements = destElements || getAll( clone ); + + for ( i = 0, l = srcElements.length; i < l; i++ ) { + cloneCopyEvent( srcElements[ i ], destElements[ i ] ); + } + } else { + cloneCopyEvent( elem, clone ); + } + } + + // Preserve script evaluation history + destElements = getAll( clone, "script" ); + if ( destElements.length > 0 ) { + setGlobalEval( destElements, !inPage && getAll( elem, "script" ) ); + } + + // Return the cloned set + return clone; + }, + + cleanData: function( elems ) { + var data, elem, type, + special = jQuery.event.special, + i = 0; + + for ( ; ( elem = elems[ i ] ) !== undefined; i++ ) { + if ( acceptData( elem ) ) { + if ( ( data = elem[ dataPriv.expando ] ) ) { + if ( data.events ) { + for ( type in data.events ) { + if ( special[ type ] ) { + jQuery.event.remove( elem, type ); + + // This is a shortcut to avoid jQuery.event.remove's overhead + } else { + jQuery.removeEvent( elem, type, data.handle ); + } + } + } + + // Support: Chrome <=35 - 45+ + // Assign undefined instead of using delete, see Data#remove + elem[ dataPriv.expando ] = undefined; + } + if ( elem[ dataUser.expando ] ) { + + // Support: Chrome <=35 - 45+ + // Assign undefined instead of using delete, see Data#remove + elem[ dataUser.expando ] = undefined; + } + } + } + } +} ); + +jQuery.fn.extend( { + detach: function( selector ) { + return remove( this, selector, true ); + }, + + remove: function( selector ) { + return remove( this, selector ); + }, + + text: function( value ) { + return access( this, function( value ) { + return value === undefined ? + jQuery.text( this ) : + this.empty().each( function() { + if ( this.nodeType === 1 || this.nodeType === 11 || this.nodeType === 9 ) { + this.textContent = value; + } + } ); + }, null, value, arguments.length ); + }, + + append: function() { + return domManip( this, arguments, function( elem ) { + if ( this.nodeType === 1 || this.nodeType === 11 || this.nodeType === 9 ) { + var target = manipulationTarget( this, elem ); + target.appendChild( elem ); + } + } ); + }, + + prepend: function() { + return domManip( this, arguments, function( elem ) { + if ( this.nodeType === 1 || this.nodeType === 11 || this.nodeType === 9 ) { + var target = manipulationTarget( this, elem ); + target.insertBefore( elem, target.firstChild ); + } + } ); + }, + + before: function() { + return domManip( this, arguments, function( elem ) { + if ( this.parentNode ) { + this.parentNode.insertBefore( elem, this ); + } + } ); + }, + + after: function() { + return domManip( this, arguments, function( elem ) { + if ( this.parentNode ) { + this.parentNode.insertBefore( elem, this.nextSibling ); + } + } ); + }, + + empty: function() { + var elem, + i = 0; + + for ( ; ( elem = this[ i ] ) != null; i++ ) { + if ( elem.nodeType === 1 ) { + + // Prevent memory leaks + jQuery.cleanData( getAll( elem, false ) ); + + // Remove any remaining nodes + elem.textContent = ""; + } + } + + return this; + }, + + clone: function( dataAndEvents, deepDataAndEvents ) { + dataAndEvents = dataAndEvents == null ? false : dataAndEvents; + deepDataAndEvents = deepDataAndEvents == null ? dataAndEvents : deepDataAndEvents; + + return this.map( function() { + return jQuery.clone( this, dataAndEvents, deepDataAndEvents ); + } ); + }, + + html: function( value ) { + return access( this, function( value ) { + var elem = this[ 0 ] || {}, + i = 0, + l = this.length; + + if ( value === undefined && elem.nodeType === 1 ) { + return elem.innerHTML; + } + + // See if we can take a shortcut and just use innerHTML + if ( typeof value === "string" && !rnoInnerhtml.test( value ) && + !wrapMap[ ( rtagName.exec( value ) || [ "", "" ] )[ 1 ].toLowerCase() ] ) { + + value = jQuery.htmlPrefilter( value ); + + try { + for ( ; i < l; i++ ) { + elem = this[ i ] || {}; + + // Remove element nodes and prevent memory leaks + if ( elem.nodeType === 1 ) { + jQuery.cleanData( getAll( elem, false ) ); + elem.innerHTML = value; + } + } + + elem = 0; + + // If using innerHTML throws an exception, use the fallback method + } catch ( e ) {} + } + + if ( elem ) { + this.empty().append( value ); + } + }, null, value, arguments.length ); + }, + + replaceWith: function() { + var ignored = []; + + // Make the changes, replacing each non-ignored context element with the new content + return domManip( this, arguments, function( elem ) { + var parent = this.parentNode; + + if ( jQuery.inArray( this, ignored ) < 0 ) { + jQuery.cleanData( getAll( this ) ); + if ( parent ) { + parent.replaceChild( elem, this ); + } + } + + // Force callback invocation + }, ignored ); + } +} ); + +jQuery.each( { + appendTo: "append", + prependTo: "prepend", + insertBefore: "before", + insertAfter: "after", + replaceAll: "replaceWith" +}, function( name, original ) { + jQuery.fn[ name ] = function( selector ) { + var elems, + ret = [], + insert = jQuery( selector ), + last = insert.length - 1, + i = 0; + + for ( ; i <= last; i++ ) { + elems = i === last ? this : this.clone( true ); + jQuery( insert[ i ] )[ original ]( elems ); + + // Support: Android <=4.0 only, PhantomJS 1 only + // .get() because push.apply(_, arraylike) throws on ancient WebKit + push.apply( ret, elems.get() ); + } + + return this.pushStack( ret ); + }; +} ); +var rnumnonpx = new RegExp( "^(" + pnum + ")(?!px)[a-z%]+$", "i" ); + +var getStyles = function( elem ) { + + // Support: IE <=11 only, Firefox <=30 (#15098, #14150) + // IE throws on elements created in popups + // FF meanwhile throws on frame elements through "defaultView.getComputedStyle" + var view = elem.ownerDocument.defaultView; + + if ( !view || !view.opener ) { + view = window; + } + + return view.getComputedStyle( elem ); + }; + +var swap = function( elem, options, callback ) { + var ret, name, + old = {}; + + // Remember the old values, and insert the new ones + for ( name in options ) { + old[ name ] = elem.style[ name ]; + elem.style[ name ] = options[ name ]; + } + + ret = callback.call( elem ); + + // Revert the old values + for ( name in options ) { + elem.style[ name ] = old[ name ]; + } + + return ret; +}; + + +var rboxStyle = new RegExp( cssExpand.join( "|" ), "i" ); + + + +( function() { + + // Executing both pixelPosition & boxSizingReliable tests require only one layout + // so they're executed at the same time to save the second computation. + function computeStyleTests() { + + // This is a singleton, we need to execute it only once + if ( !div ) { + return; + } + + container.style.cssText = "position:absolute;left:-11111px;width:60px;" + + "margin-top:1px;padding:0;border:0"; + div.style.cssText = + "position:relative;display:block;box-sizing:border-box;overflow:scroll;" + + "margin:auto;border:1px;padding:1px;" + + "width:60%;top:1%"; + documentElement.appendChild( container ).appendChild( div ); + + var divStyle = window.getComputedStyle( div ); + pixelPositionVal = divStyle.top !== "1%"; + + // Support: Android 4.0 - 4.3 only, Firefox <=3 - 44 + reliableMarginLeftVal = roundPixelMeasures( divStyle.marginLeft ) === 12; + + // Support: Android 4.0 - 4.3 only, Safari <=9.1 - 10.1, iOS <=7.0 - 9.3 + // Some styles come back with percentage values, even though they shouldn't + div.style.right = "60%"; + pixelBoxStylesVal = roundPixelMeasures( divStyle.right ) === 36; + + // Support: IE 9 - 11 only + // Detect misreporting of content dimensions for box-sizing:border-box elements + boxSizingReliableVal = roundPixelMeasures( divStyle.width ) === 36; + + // Support: IE 9 only + // Detect overflow:scroll screwiness (gh-3699) + // Support: Chrome <=64 + // Don't get tricked when zoom affects offsetWidth (gh-4029) + div.style.position = "absolute"; + scrollboxSizeVal = roundPixelMeasures( div.offsetWidth / 3 ) === 12; + + documentElement.removeChild( container ); + + // Nullify the div so it wouldn't be stored in the memory and + // it will also be a sign that checks already performed + div = null; + } + + function roundPixelMeasures( measure ) { + return Math.round( parseFloat( measure ) ); + } + + var pixelPositionVal, boxSizingReliableVal, scrollboxSizeVal, pixelBoxStylesVal, + reliableTrDimensionsVal, reliableMarginLeftVal, + container = document.createElement( "div" ), + div = document.createElement( "div" ); + + // Finish early in limited (non-browser) environments + if ( !div.style ) { + return; + } + + // Support: IE <=9 - 11 only + // Style of cloned element affects source element cloned (#8908) + div.style.backgroundClip = "content-box"; + div.cloneNode( true ).style.backgroundClip = ""; + support.clearCloneStyle = div.style.backgroundClip === "content-box"; + + jQuery.extend( support, { + boxSizingReliable: function() { + computeStyleTests(); + return boxSizingReliableVal; + }, + pixelBoxStyles: function() { + computeStyleTests(); + return pixelBoxStylesVal; + }, + pixelPosition: function() { + computeStyleTests(); + return pixelPositionVal; + }, + reliableMarginLeft: function() { + computeStyleTests(); + return reliableMarginLeftVal; + }, + scrollboxSize: function() { + computeStyleTests(); + return scrollboxSizeVal; + }, + + // Support: IE 9 - 11+, Edge 15 - 18+ + // IE/Edge misreport `getComputedStyle` of table rows with width/height + // set in CSS while `offset*` properties report correct values. + // Behavior in IE 9 is more subtle than in newer versions & it passes + // some versions of this test; make sure not to make it pass there! + // + // Support: Firefox 70+ + // Only Firefox includes border widths + // in computed dimensions. (gh-4529) + reliableTrDimensions: function() { + var table, tr, trChild, trStyle; + if ( reliableTrDimensionsVal == null ) { + table = document.createElement( "table" ); + tr = document.createElement( "tr" ); + trChild = document.createElement( "div" ); + + table.style.cssText = "position:absolute;left:-11111px;border-collapse:separate"; + tr.style.cssText = "border:1px solid"; + + // Support: Chrome 86+ + // Height set through cssText does not get applied. + // Computed height then comes back as 0. + tr.style.height = "1px"; + trChild.style.height = "9px"; + + // Support: Android 8 Chrome 86+ + // In our bodyBackground.html iframe, + // display for all div elements is set to "inline", + // which causes a problem only in Android 8 Chrome 86. + // Ensuring the div is display: block + // gets around this issue. + trChild.style.display = "block"; + + documentElement + .appendChild( table ) + .appendChild( tr ) + .appendChild( trChild ); + + trStyle = window.getComputedStyle( tr ); + reliableTrDimensionsVal = ( parseInt( trStyle.height, 10 ) + + parseInt( trStyle.borderTopWidth, 10 ) + + parseInt( trStyle.borderBottomWidth, 10 ) ) === tr.offsetHeight; + + documentElement.removeChild( table ); + } + return reliableTrDimensionsVal; + } + } ); +} )(); + + +function curCSS( elem, name, computed ) { + var width, minWidth, maxWidth, ret, + + // Support: Firefox 51+ + // Retrieving style before computed somehow + // fixes an issue with getting wrong values + // on detached elements + style = elem.style; + + computed = computed || getStyles( elem ); + + // getPropertyValue is needed for: + // .css('filter') (IE 9 only, #12537) + // .css('--customProperty) (#3144) + if ( computed ) { + ret = computed.getPropertyValue( name ) || computed[ name ]; + + if ( ret === "" && !isAttached( elem ) ) { + ret = jQuery.style( elem, name ); + } + + // A tribute to the "awesome hack by Dean Edwards" + // Android Browser returns percentage for some values, + // but width seems to be reliably pixels. + // This is against the CSSOM draft spec: + // https://drafts.csswg.org/cssom/#resolved-values + if ( !support.pixelBoxStyles() && rnumnonpx.test( ret ) && rboxStyle.test( name ) ) { + + // Remember the original values + width = style.width; + minWidth = style.minWidth; + maxWidth = style.maxWidth; + + // Put in the new values to get a computed value out + style.minWidth = style.maxWidth = style.width = ret; + ret = computed.width; + + // Revert the changed values + style.width = width; + style.minWidth = minWidth; + style.maxWidth = maxWidth; + } + } + + return ret !== undefined ? + + // Support: IE <=9 - 11 only + // IE returns zIndex value as an integer. + ret + "" : + ret; +} + + +function addGetHookIf( conditionFn, hookFn ) { + + // Define the hook, we'll check on the first run if it's really needed. + return { + get: function() { + if ( conditionFn() ) { + + // Hook not needed (or it's not possible to use it due + // to missing dependency), remove it. + delete this.get; + return; + } + + // Hook needed; redefine it so that the support test is not executed again. + return ( this.get = hookFn ).apply( this, arguments ); + } + }; +} + + +var cssPrefixes = [ "Webkit", "Moz", "ms" ], + emptyStyle = document.createElement( "div" ).style, + vendorProps = {}; + +// Return a vendor-prefixed property or undefined +function vendorPropName( name ) { + + // Check for vendor prefixed names + var capName = name[ 0 ].toUpperCase() + name.slice( 1 ), + i = cssPrefixes.length; + + while ( i-- ) { + name = cssPrefixes[ i ] + capName; + if ( name in emptyStyle ) { + return name; + } + } +} + +// Return a potentially-mapped jQuery.cssProps or vendor prefixed property +function finalPropName( name ) { + var final = jQuery.cssProps[ name ] || vendorProps[ name ]; + + if ( final ) { + return final; + } + if ( name in emptyStyle ) { + return name; + } + return vendorProps[ name ] = vendorPropName( name ) || name; +} + + +var + + // Swappable if display is none or starts with table + // except "table", "table-cell", or "table-caption" + // See here for display values: https://developer.mozilla.org/en-US/docs/CSS/display + rdisplayswap = /^(none|table(?!-c[ea]).+)/, + rcustomProp = /^--/, + cssShow = { position: "absolute", visibility: "hidden", display: "block" }, + cssNormalTransform = { + letterSpacing: "0", + fontWeight: "400" + }; + +function setPositiveNumber( _elem, value, subtract ) { + + // Any relative (+/-) values have already been + // normalized at this point + var matches = rcssNum.exec( value ); + return matches ? + + // Guard against undefined "subtract", e.g., when used as in cssHooks + Math.max( 0, matches[ 2 ] - ( subtract || 0 ) ) + ( matches[ 3 ] || "px" ) : + value; +} + +function boxModelAdjustment( elem, dimension, box, isBorderBox, styles, computedVal ) { + var i = dimension === "width" ? 1 : 0, + extra = 0, + delta = 0; + + // Adjustment may not be necessary + if ( box === ( isBorderBox ? "border" : "content" ) ) { + return 0; + } + + for ( ; i < 4; i += 2 ) { + + // Both box models exclude margin + if ( box === "margin" ) { + delta += jQuery.css( elem, box + cssExpand[ i ], true, styles ); + } + + // If we get here with a content-box, we're seeking "padding" or "border" or "margin" + if ( !isBorderBox ) { + + // Add padding + delta += jQuery.css( elem, "padding" + cssExpand[ i ], true, styles ); + + // For "border" or "margin", add border + if ( box !== "padding" ) { + delta += jQuery.css( elem, "border" + cssExpand[ i ] + "Width", true, styles ); + + // But still keep track of it otherwise + } else { + extra += jQuery.css( elem, "border" + cssExpand[ i ] + "Width", true, styles ); + } + + // If we get here with a border-box (content + padding + border), we're seeking "content" or + // "padding" or "margin" + } else { + + // For "content", subtract padding + if ( box === "content" ) { + delta -= jQuery.css( elem, "padding" + cssExpand[ i ], true, styles ); + } + + // For "content" or "padding", subtract border + if ( box !== "margin" ) { + delta -= jQuery.css( elem, "border" + cssExpand[ i ] + "Width", true, styles ); + } + } + } + + // Account for positive content-box scroll gutter when requested by providing computedVal + if ( !isBorderBox && computedVal >= 0 ) { + + // offsetWidth/offsetHeight is a rounded sum of content, padding, scroll gutter, and border + // Assuming integer scroll gutter, subtract the rest and round down + delta += Math.max( 0, Math.ceil( + elem[ "offset" + dimension[ 0 ].toUpperCase() + dimension.slice( 1 ) ] - + computedVal - + delta - + extra - + 0.5 + + // If offsetWidth/offsetHeight is unknown, then we can't determine content-box scroll gutter + // Use an explicit zero to avoid NaN (gh-3964) + ) ) || 0; + } + + return delta; +} + +function getWidthOrHeight( elem, dimension, extra ) { + + // Start with computed style + var styles = getStyles( elem ), + + // To avoid forcing a reflow, only fetch boxSizing if we need it (gh-4322). + // Fake content-box until we know it's needed to know the true value. + boxSizingNeeded = !support.boxSizingReliable() || extra, + isBorderBox = boxSizingNeeded && + jQuery.css( elem, "boxSizing", false, styles ) === "border-box", + valueIsBorderBox = isBorderBox, + + val = curCSS( elem, dimension, styles ), + offsetProp = "offset" + dimension[ 0 ].toUpperCase() + dimension.slice( 1 ); + + // Support: Firefox <=54 + // Return a confounding non-pixel value or feign ignorance, as appropriate. + if ( rnumnonpx.test( val ) ) { + if ( !extra ) { + return val; + } + val = "auto"; + } + + + // Support: IE 9 - 11 only + // Use offsetWidth/offsetHeight for when box sizing is unreliable. + // In those cases, the computed value can be trusted to be border-box. + if ( ( !support.boxSizingReliable() && isBorderBox || + + // Support: IE 10 - 11+, Edge 15 - 18+ + // IE/Edge misreport `getComputedStyle` of table rows with width/height + // set in CSS while `offset*` properties report correct values. + // Interestingly, in some cases IE 9 doesn't suffer from this issue. + !support.reliableTrDimensions() && nodeName( elem, "tr" ) || + + // Fall back to offsetWidth/offsetHeight when value is "auto" + // This happens for inline elements with no explicit setting (gh-3571) + val === "auto" || + + // Support: Android <=4.1 - 4.3 only + // Also use offsetWidth/offsetHeight for misreported inline dimensions (gh-3602) + !parseFloat( val ) && jQuery.css( elem, "display", false, styles ) === "inline" ) && + + // Make sure the element is visible & connected + elem.getClientRects().length ) { + + isBorderBox = jQuery.css( elem, "boxSizing", false, styles ) === "border-box"; + + // Where available, offsetWidth/offsetHeight approximate border box dimensions. + // Where not available (e.g., SVG), assume unreliable box-sizing and interpret the + // retrieved value as a content box dimension. + valueIsBorderBox = offsetProp in elem; + if ( valueIsBorderBox ) { + val = elem[ offsetProp ]; + } + } + + // Normalize "" and auto + val = parseFloat( val ) || 0; + + // Adjust for the element's box model + return ( val + + boxModelAdjustment( + elem, + dimension, + extra || ( isBorderBox ? "border" : "content" ), + valueIsBorderBox, + styles, + + // Provide the current computed size to request scroll gutter calculation (gh-3589) + val + ) + ) + "px"; +} + +jQuery.extend( { + + // Add in style property hooks for overriding the default + // behavior of getting and setting a style property + cssHooks: { + opacity: { + get: function( elem, computed ) { + if ( computed ) { + + // We should always get a number back from opacity + var ret = curCSS( elem, "opacity" ); + return ret === "" ? "1" : ret; + } + } + } + }, + + // Don't automatically add "px" to these possibly-unitless properties + cssNumber: { + "animationIterationCount": true, + "columnCount": true, + "fillOpacity": true, + "flexGrow": true, + "flexShrink": true, + "fontWeight": true, + "gridArea": true, + "gridColumn": true, + "gridColumnEnd": true, + "gridColumnStart": true, + "gridRow": true, + "gridRowEnd": true, + "gridRowStart": true, + "lineHeight": true, + "opacity": true, + "order": true, + "orphans": true, + "widows": true, + "zIndex": true, + "zoom": true + }, + + // Add in properties whose names you wish to fix before + // setting or getting the value + cssProps: {}, + + // Get and set the style property on a DOM Node + style: function( elem, name, value, extra ) { + + // Don't set styles on text and comment nodes + if ( !elem || elem.nodeType === 3 || elem.nodeType === 8 || !elem.style ) { + return; + } + + // Make sure that we're working with the right name + var ret, type, hooks, + origName = camelCase( name ), + isCustomProp = rcustomProp.test( name ), + style = elem.style; + + // Make sure that we're working with the right name. We don't + // want to query the value if it is a CSS custom property + // since they are user-defined. + if ( !isCustomProp ) { + name = finalPropName( origName ); + } + + // Gets hook for the prefixed version, then unprefixed version + hooks = jQuery.cssHooks[ name ] || jQuery.cssHooks[ origName ]; + + // Check if we're setting a value + if ( value !== undefined ) { + type = typeof value; + + // Convert "+=" or "-=" to relative numbers (#7345) + if ( type === "string" && ( ret = rcssNum.exec( value ) ) && ret[ 1 ] ) { + value = adjustCSS( elem, name, ret ); + + // Fixes bug #9237 + type = "number"; + } + + // Make sure that null and NaN values aren't set (#7116) + if ( value == null || value !== value ) { + return; + } + + // If a number was passed in, add the unit (except for certain CSS properties) + // The isCustomProp check can be removed in jQuery 4.0 when we only auto-append + // "px" to a few hardcoded values. + if ( type === "number" && !isCustomProp ) { + value += ret && ret[ 3 ] || ( jQuery.cssNumber[ origName ] ? "" : "px" ); + } + + // background-* props affect original clone's values + if ( !support.clearCloneStyle && value === "" && name.indexOf( "background" ) === 0 ) { + style[ name ] = "inherit"; + } + + // If a hook was provided, use that value, otherwise just set the specified value + if ( !hooks || !( "set" in hooks ) || + ( value = hooks.set( elem, value, extra ) ) !== undefined ) { + + if ( isCustomProp ) { + style.setProperty( name, value ); + } else { + style[ name ] = value; + } + } + + } else { + + // If a hook was provided get the non-computed value from there + if ( hooks && "get" in hooks && + ( ret = hooks.get( elem, false, extra ) ) !== undefined ) { + + return ret; + } + + // Otherwise just get the value from the style object + return style[ name ]; + } + }, + + css: function( elem, name, extra, styles ) { + var val, num, hooks, + origName = camelCase( name ), + isCustomProp = rcustomProp.test( name ); + + // Make sure that we're working with the right name. We don't + // want to modify the value if it is a CSS custom property + // since they are user-defined. + if ( !isCustomProp ) { + name = finalPropName( origName ); + } + + // Try prefixed name followed by the unprefixed name + hooks = jQuery.cssHooks[ name ] || jQuery.cssHooks[ origName ]; + + // If a hook was provided get the computed value from there + if ( hooks && "get" in hooks ) { + val = hooks.get( elem, true, extra ); + } + + // Otherwise, if a way to get the computed value exists, use that + if ( val === undefined ) { + val = curCSS( elem, name, styles ); + } + + // Convert "normal" to computed value + if ( val === "normal" && name in cssNormalTransform ) { + val = cssNormalTransform[ name ]; + } + + // Make numeric if forced or a qualifier was provided and val looks numeric + if ( extra === "" || extra ) { + num = parseFloat( val ); + return extra === true || isFinite( num ) ? num || 0 : val; + } + + return val; + } +} ); + +jQuery.each( [ "height", "width" ], function( _i, dimension ) { + jQuery.cssHooks[ dimension ] = { + get: function( elem, computed, extra ) { + if ( computed ) { + + // Certain elements can have dimension info if we invisibly show them + // but it must have a current display style that would benefit + return rdisplayswap.test( jQuery.css( elem, "display" ) ) && + + // Support: Safari 8+ + // Table columns in Safari have non-zero offsetWidth & zero + // getBoundingClientRect().width unless display is changed. + // Support: IE <=11 only + // Running getBoundingClientRect on a disconnected node + // in IE throws an error. + ( !elem.getClientRects().length || !elem.getBoundingClientRect().width ) ? + swap( elem, cssShow, function() { + return getWidthOrHeight( elem, dimension, extra ); + } ) : + getWidthOrHeight( elem, dimension, extra ); + } + }, + + set: function( elem, value, extra ) { + var matches, + styles = getStyles( elem ), + + // Only read styles.position if the test has a chance to fail + // to avoid forcing a reflow. + scrollboxSizeBuggy = !support.scrollboxSize() && + styles.position === "absolute", + + // To avoid forcing a reflow, only fetch boxSizing if we need it (gh-3991) + boxSizingNeeded = scrollboxSizeBuggy || extra, + isBorderBox = boxSizingNeeded && + jQuery.css( elem, "boxSizing", false, styles ) === "border-box", + subtract = extra ? + boxModelAdjustment( + elem, + dimension, + extra, + isBorderBox, + styles + ) : + 0; + + // Account for unreliable border-box dimensions by comparing offset* to computed and + // faking a content-box to get border and padding (gh-3699) + if ( isBorderBox && scrollboxSizeBuggy ) { + subtract -= Math.ceil( + elem[ "offset" + dimension[ 0 ].toUpperCase() + dimension.slice( 1 ) ] - + parseFloat( styles[ dimension ] ) - + boxModelAdjustment( elem, dimension, "border", false, styles ) - + 0.5 + ); + } + + // Convert to pixels if value adjustment is needed + if ( subtract && ( matches = rcssNum.exec( value ) ) && + ( matches[ 3 ] || "px" ) !== "px" ) { + + elem.style[ dimension ] = value; + value = jQuery.css( elem, dimension ); + } + + return setPositiveNumber( elem, value, subtract ); + } + }; +} ); + +jQuery.cssHooks.marginLeft = addGetHookIf( support.reliableMarginLeft, + function( elem, computed ) { + if ( computed ) { + return ( parseFloat( curCSS( elem, "marginLeft" ) ) || + elem.getBoundingClientRect().left - + swap( elem, { marginLeft: 0 }, function() { + return elem.getBoundingClientRect().left; + } ) + ) + "px"; + } + } +); + +// These hooks are used by animate to expand properties +jQuery.each( { + margin: "", + padding: "", + border: "Width" +}, function( prefix, suffix ) { + jQuery.cssHooks[ prefix + suffix ] = { + expand: function( value ) { + var i = 0, + expanded = {}, + + // Assumes a single number if not a string + parts = typeof value === "string" ? value.split( " " ) : [ value ]; + + for ( ; i < 4; i++ ) { + expanded[ prefix + cssExpand[ i ] + suffix ] = + parts[ i ] || parts[ i - 2 ] || parts[ 0 ]; + } + + return expanded; + } + }; + + if ( prefix !== "margin" ) { + jQuery.cssHooks[ prefix + suffix ].set = setPositiveNumber; + } +} ); + +jQuery.fn.extend( { + css: function( name, value ) { + return access( this, function( elem, name, value ) { + var styles, len, + map = {}, + i = 0; + + if ( Array.isArray( name ) ) { + styles = getStyles( elem ); + len = name.length; + + for ( ; i < len; i++ ) { + map[ name[ i ] ] = jQuery.css( elem, name[ i ], false, styles ); + } + + return map; + } + + return value !== undefined ? + jQuery.style( elem, name, value ) : + jQuery.css( elem, name ); + }, name, value, arguments.length > 1 ); + } +} ); + + +function Tween( elem, options, prop, end, easing ) { + return new Tween.prototype.init( elem, options, prop, end, easing ); +} +jQuery.Tween = Tween; + +Tween.prototype = { + constructor: Tween, + init: function( elem, options, prop, end, easing, unit ) { + this.elem = elem; + this.prop = prop; + this.easing = easing || jQuery.easing._default; + this.options = options; + this.start = this.now = this.cur(); + this.end = end; + this.unit = unit || ( jQuery.cssNumber[ prop ] ? "" : "px" ); + }, + cur: function() { + var hooks = Tween.propHooks[ this.prop ]; + + return hooks && hooks.get ? + hooks.get( this ) : + Tween.propHooks._default.get( this ); + }, + run: function( percent ) { + var eased, + hooks = Tween.propHooks[ this.prop ]; + + if ( this.options.duration ) { + this.pos = eased = jQuery.easing[ this.easing ]( + percent, this.options.duration * percent, 0, 1, this.options.duration + ); + } else { + this.pos = eased = percent; + } + this.now = ( this.end - this.start ) * eased + this.start; + + if ( this.options.step ) { + this.options.step.call( this.elem, this.now, this ); + } + + if ( hooks && hooks.set ) { + hooks.set( this ); + } else { + Tween.propHooks._default.set( this ); + } + return this; + } +}; + +Tween.prototype.init.prototype = Tween.prototype; + +Tween.propHooks = { + _default: { + get: function( tween ) { + var result; + + // Use a property on the element directly when it is not a DOM element, + // or when there is no matching style property that exists. + if ( tween.elem.nodeType !== 1 || + tween.elem[ tween.prop ] != null && tween.elem.style[ tween.prop ] == null ) { + return tween.elem[ tween.prop ]; + } + + // Passing an empty string as a 3rd parameter to .css will automatically + // attempt a parseFloat and fallback to a string if the parse fails. + // Simple values such as "10px" are parsed to Float; + // complex values such as "rotate(1rad)" are returned as-is. + result = jQuery.css( tween.elem, tween.prop, "" ); + + // Empty strings, null, undefined and "auto" are converted to 0. + return !result || result === "auto" ? 0 : result; + }, + set: function( tween ) { + + // Use step hook for back compat. + // Use cssHook if its there. + // Use .style if available and use plain properties where available. + if ( jQuery.fx.step[ tween.prop ] ) { + jQuery.fx.step[ tween.prop ]( tween ); + } else if ( tween.elem.nodeType === 1 && ( + jQuery.cssHooks[ tween.prop ] || + tween.elem.style[ finalPropName( tween.prop ) ] != null ) ) { + jQuery.style( tween.elem, tween.prop, tween.now + tween.unit ); + } else { + tween.elem[ tween.prop ] = tween.now; + } + } + } +}; + +// Support: IE <=9 only +// Panic based approach to setting things on disconnected nodes +Tween.propHooks.scrollTop = Tween.propHooks.scrollLeft = { + set: function( tween ) { + if ( tween.elem.nodeType && tween.elem.parentNode ) { + tween.elem[ tween.prop ] = tween.now; + } + } +}; + +jQuery.easing = { + linear: function( p ) { + return p; + }, + swing: function( p ) { + return 0.5 - Math.cos( p * Math.PI ) / 2; + }, + _default: "swing" +}; + +jQuery.fx = Tween.prototype.init; + +// Back compat <1.8 extension point +jQuery.fx.step = {}; + + + + +var + fxNow, inProgress, + rfxtypes = /^(?:toggle|show|hide)$/, + rrun = /queueHooks$/; + +function schedule() { + if ( inProgress ) { + if ( document.hidden === false && window.requestAnimationFrame ) { + window.requestAnimationFrame( schedule ); + } else { + window.setTimeout( schedule, jQuery.fx.interval ); + } + + jQuery.fx.tick(); + } +} + +// Animations created synchronously will run synchronously +function createFxNow() { + window.setTimeout( function() { + fxNow = undefined; + } ); + return ( fxNow = Date.now() ); +} + +// Generate parameters to create a standard animation +function genFx( type, includeWidth ) { + var which, + i = 0, + attrs = { height: type }; + + // If we include width, step value is 1 to do all cssExpand values, + // otherwise step value is 2 to skip over Left and Right + includeWidth = includeWidth ? 1 : 0; + for ( ; i < 4; i += 2 - includeWidth ) { + which = cssExpand[ i ]; + attrs[ "margin" + which ] = attrs[ "padding" + which ] = type; + } + + if ( includeWidth ) { + attrs.opacity = attrs.width = type; + } + + return attrs; +} + +function createTween( value, prop, animation ) { + var tween, + collection = ( Animation.tweeners[ prop ] || [] ).concat( Animation.tweeners[ "*" ] ), + index = 0, + length = collection.length; + for ( ; index < length; index++ ) { + if ( ( tween = collection[ index ].call( animation, prop, value ) ) ) { + + // We're done with this property + return tween; + } + } +} + +function defaultPrefilter( elem, props, opts ) { + var prop, value, toggle, hooks, oldfire, propTween, restoreDisplay, display, + isBox = "width" in props || "height" in props, + anim = this, + orig = {}, + style = elem.style, + hidden = elem.nodeType && isHiddenWithinTree( elem ), + dataShow = dataPriv.get( elem, "fxshow" ); + + // Queue-skipping animations hijack the fx hooks + if ( !opts.queue ) { + hooks = jQuery._queueHooks( elem, "fx" ); + if ( hooks.unqueued == null ) { + hooks.unqueued = 0; + oldfire = hooks.empty.fire; + hooks.empty.fire = function() { + if ( !hooks.unqueued ) { + oldfire(); + } + }; + } + hooks.unqueued++; + + anim.always( function() { + + // Ensure the complete handler is called before this completes + anim.always( function() { + hooks.unqueued--; + if ( !jQuery.queue( elem, "fx" ).length ) { + hooks.empty.fire(); + } + } ); + } ); + } + + // Detect show/hide animations + for ( prop in props ) { + value = props[ prop ]; + if ( rfxtypes.test( value ) ) { + delete props[ prop ]; + toggle = toggle || value === "toggle"; + if ( value === ( hidden ? "hide" : "show" ) ) { + + // Pretend to be hidden if this is a "show" and + // there is still data from a stopped show/hide + if ( value === "show" && dataShow && dataShow[ prop ] !== undefined ) { + hidden = true; + + // Ignore all other no-op show/hide data + } else { + continue; + } + } + orig[ prop ] = dataShow && dataShow[ prop ] || jQuery.style( elem, prop ); + } + } + + // Bail out if this is a no-op like .hide().hide() + propTween = !jQuery.isEmptyObject( props ); + if ( !propTween && jQuery.isEmptyObject( orig ) ) { + return; + } + + // Restrict "overflow" and "display" styles during box animations + if ( isBox && elem.nodeType === 1 ) { + + // Support: IE <=9 - 11, Edge 12 - 15 + // Record all 3 overflow attributes because IE does not infer the shorthand + // from identically-valued overflowX and overflowY and Edge just mirrors + // the overflowX value there. + opts.overflow = [ style.overflow, style.overflowX, style.overflowY ]; + + // Identify a display type, preferring old show/hide data over the CSS cascade + restoreDisplay = dataShow && dataShow.display; + if ( restoreDisplay == null ) { + restoreDisplay = dataPriv.get( elem, "display" ); + } + display = jQuery.css( elem, "display" ); + if ( display === "none" ) { + if ( restoreDisplay ) { + display = restoreDisplay; + } else { + + // Get nonempty value(s) by temporarily forcing visibility + showHide( [ elem ], true ); + restoreDisplay = elem.style.display || restoreDisplay; + display = jQuery.css( elem, "display" ); + showHide( [ elem ] ); + } + } + + // Animate inline elements as inline-block + if ( display === "inline" || display === "inline-block" && restoreDisplay != null ) { + if ( jQuery.css( elem, "float" ) === "none" ) { + + // Restore the original display value at the end of pure show/hide animations + if ( !propTween ) { + anim.done( function() { + style.display = restoreDisplay; + } ); + if ( restoreDisplay == null ) { + display = style.display; + restoreDisplay = display === "none" ? "" : display; + } + } + style.display = "inline-block"; + } + } + } + + if ( opts.overflow ) { + style.overflow = "hidden"; + anim.always( function() { + style.overflow = opts.overflow[ 0 ]; + style.overflowX = opts.overflow[ 1 ]; + style.overflowY = opts.overflow[ 2 ]; + } ); + } + + // Implement show/hide animations + propTween = false; + for ( prop in orig ) { + + // General show/hide setup for this element animation + if ( !propTween ) { + if ( dataShow ) { + if ( "hidden" in dataShow ) { + hidden = dataShow.hidden; + } + } else { + dataShow = dataPriv.access( elem, "fxshow", { display: restoreDisplay } ); + } + + // Store hidden/visible for toggle so `.stop().toggle()` "reverses" + if ( toggle ) { + dataShow.hidden = !hidden; + } + + // Show elements before animating them + if ( hidden ) { + showHide( [ elem ], true ); + } + + /* eslint-disable no-loop-func */ + + anim.done( function() { + + /* eslint-enable no-loop-func */ + + // The final step of a "hide" animation is actually hiding the element + if ( !hidden ) { + showHide( [ elem ] ); + } + dataPriv.remove( elem, "fxshow" ); + for ( prop in orig ) { + jQuery.style( elem, prop, orig[ prop ] ); + } + } ); + } + + // Per-property setup + propTween = createTween( hidden ? dataShow[ prop ] : 0, prop, anim ); + if ( !( prop in dataShow ) ) { + dataShow[ prop ] = propTween.start; + if ( hidden ) { + propTween.end = propTween.start; + propTween.start = 0; + } + } + } +} + +function propFilter( props, specialEasing ) { + var index, name, easing, value, hooks; + + // camelCase, specialEasing and expand cssHook pass + for ( index in props ) { + name = camelCase( index ); + easing = specialEasing[ name ]; + value = props[ index ]; + if ( Array.isArray( value ) ) { + easing = value[ 1 ]; + value = props[ index ] = value[ 0 ]; + } + + if ( index !== name ) { + props[ name ] = value; + delete props[ index ]; + } + + hooks = jQuery.cssHooks[ name ]; + if ( hooks && "expand" in hooks ) { + value = hooks.expand( value ); + delete props[ name ]; + + // Not quite $.extend, this won't overwrite existing keys. + // Reusing 'index' because we have the correct "name" + for ( index in value ) { + if ( !( index in props ) ) { + props[ index ] = value[ index ]; + specialEasing[ index ] = easing; + } + } + } else { + specialEasing[ name ] = easing; + } + } +} + +function Animation( elem, properties, options ) { + var result, + stopped, + index = 0, + length = Animation.prefilters.length, + deferred = jQuery.Deferred().always( function() { + + // Don't match elem in the :animated selector + delete tick.elem; + } ), + tick = function() { + if ( stopped ) { + return false; + } + var currentTime = fxNow || createFxNow(), + remaining = Math.max( 0, animation.startTime + animation.duration - currentTime ), + + // Support: Android 2.3 only + // Archaic crash bug won't allow us to use `1 - ( 0.5 || 0 )` (#12497) + temp = remaining / animation.duration || 0, + percent = 1 - temp, + index = 0, + length = animation.tweens.length; + + for ( ; index < length; index++ ) { + animation.tweens[ index ].run( percent ); + } + + deferred.notifyWith( elem, [ animation, percent, remaining ] ); + + // If there's more to do, yield + if ( percent < 1 && length ) { + return remaining; + } + + // If this was an empty animation, synthesize a final progress notification + if ( !length ) { + deferred.notifyWith( elem, [ animation, 1, 0 ] ); + } + + // Resolve the animation and report its conclusion + deferred.resolveWith( elem, [ animation ] ); + return false; + }, + animation = deferred.promise( { + elem: elem, + props: jQuery.extend( {}, properties ), + opts: jQuery.extend( true, { + specialEasing: {}, + easing: jQuery.easing._default + }, options ), + originalProperties: properties, + originalOptions: options, + startTime: fxNow || createFxNow(), + duration: options.duration, + tweens: [], + createTween: function( prop, end ) { + var tween = jQuery.Tween( elem, animation.opts, prop, end, + animation.opts.specialEasing[ prop ] || animation.opts.easing ); + animation.tweens.push( tween ); + return tween; + }, + stop: function( gotoEnd ) { + var index = 0, + + // If we are going to the end, we want to run all the tweens + // otherwise we skip this part + length = gotoEnd ? animation.tweens.length : 0; + if ( stopped ) { + return this; + } + stopped = true; + for ( ; index < length; index++ ) { + animation.tweens[ index ].run( 1 ); + } + + // Resolve when we played the last frame; otherwise, reject + if ( gotoEnd ) { + deferred.notifyWith( elem, [ animation, 1, 0 ] ); + deferred.resolveWith( elem, [ animation, gotoEnd ] ); + } else { + deferred.rejectWith( elem, [ animation, gotoEnd ] ); + } + return this; + } + } ), + props = animation.props; + + propFilter( props, animation.opts.specialEasing ); + + for ( ; index < length; index++ ) { + result = Animation.prefilters[ index ].call( animation, elem, props, animation.opts ); + if ( result ) { + if ( isFunction( result.stop ) ) { + jQuery._queueHooks( animation.elem, animation.opts.queue ).stop = + result.stop.bind( result ); + } + return result; + } + } + + jQuery.map( props, createTween, animation ); + + if ( isFunction( animation.opts.start ) ) { + animation.opts.start.call( elem, animation ); + } + + // Attach callbacks from options + animation + .progress( animation.opts.progress ) + .done( animation.opts.done, animation.opts.complete ) + .fail( animation.opts.fail ) + .always( animation.opts.always ); + + jQuery.fx.timer( + jQuery.extend( tick, { + elem: elem, + anim: animation, + queue: animation.opts.queue + } ) + ); + + return animation; +} + +jQuery.Animation = jQuery.extend( Animation, { + + tweeners: { + "*": [ function( prop, value ) { + var tween = this.createTween( prop, value ); + adjustCSS( tween.elem, prop, rcssNum.exec( value ), tween ); + return tween; + } ] + }, + + tweener: function( props, callback ) { + if ( isFunction( props ) ) { + callback = props; + props = [ "*" ]; + } else { + props = props.match( rnothtmlwhite ); + } + + var prop, + index = 0, + length = props.length; + + for ( ; index < length; index++ ) { + prop = props[ index ]; + Animation.tweeners[ prop ] = Animation.tweeners[ prop ] || []; + Animation.tweeners[ prop ].unshift( callback ); + } + }, + + prefilters: [ defaultPrefilter ], + + prefilter: function( callback, prepend ) { + if ( prepend ) { + Animation.prefilters.unshift( callback ); + } else { + Animation.prefilters.push( callback ); + } + } +} ); + +jQuery.speed = function( speed, easing, fn ) { + var opt = speed && typeof speed === "object" ? jQuery.extend( {}, speed ) : { + complete: fn || !fn && easing || + isFunction( speed ) && speed, + duration: speed, + easing: fn && easing || easing && !isFunction( easing ) && easing + }; + + // Go to the end state if fx are off + if ( jQuery.fx.off ) { + opt.duration = 0; + + } else { + if ( typeof opt.duration !== "number" ) { + if ( opt.duration in jQuery.fx.speeds ) { + opt.duration = jQuery.fx.speeds[ opt.duration ]; + + } else { + opt.duration = jQuery.fx.speeds._default; + } + } + } + + // Normalize opt.queue - true/undefined/null -> "fx" + if ( opt.queue == null || opt.queue === true ) { + opt.queue = "fx"; + } + + // Queueing + opt.old = opt.complete; + + opt.complete = function() { + if ( isFunction( opt.old ) ) { + opt.old.call( this ); + } + + if ( opt.queue ) { + jQuery.dequeue( this, opt.queue ); + } + }; + + return opt; +}; + +jQuery.fn.extend( { + fadeTo: function( speed, to, easing, callback ) { + + // Show any hidden elements after setting opacity to 0 + return this.filter( isHiddenWithinTree ).css( "opacity", 0 ).show() + + // Animate to the value specified + .end().animate( { opacity: to }, speed, easing, callback ); + }, + animate: function( prop, speed, easing, callback ) { + var empty = jQuery.isEmptyObject( prop ), + optall = jQuery.speed( speed, easing, callback ), + doAnimation = function() { + + // Operate on a copy of prop so per-property easing won't be lost + var anim = Animation( this, jQuery.extend( {}, prop ), optall ); + + // Empty animations, or finishing resolves immediately + if ( empty || dataPriv.get( this, "finish" ) ) { + anim.stop( true ); + } + }; + + doAnimation.finish = doAnimation; + + return empty || optall.queue === false ? + this.each( doAnimation ) : + this.queue( optall.queue, doAnimation ); + }, + stop: function( type, clearQueue, gotoEnd ) { + var stopQueue = function( hooks ) { + var stop = hooks.stop; + delete hooks.stop; + stop( gotoEnd ); + }; + + if ( typeof type !== "string" ) { + gotoEnd = clearQueue; + clearQueue = type; + type = undefined; + } + if ( clearQueue ) { + this.queue( type || "fx", [] ); + } + + return this.each( function() { + var dequeue = true, + index = type != null && type + "queueHooks", + timers = jQuery.timers, + data = dataPriv.get( this ); + + if ( index ) { + if ( data[ index ] && data[ index ].stop ) { + stopQueue( data[ index ] ); + } + } else { + for ( index in data ) { + if ( data[ index ] && data[ index ].stop && rrun.test( index ) ) { + stopQueue( data[ index ] ); + } + } + } + + for ( index = timers.length; index--; ) { + if ( timers[ index ].elem === this && + ( type == null || timers[ index ].queue === type ) ) { + + timers[ index ].anim.stop( gotoEnd ); + dequeue = false; + timers.splice( index, 1 ); + } + } + + // Start the next in the queue if the last step wasn't forced. + // Timers currently will call their complete callbacks, which + // will dequeue but only if they were gotoEnd. + if ( dequeue || !gotoEnd ) { + jQuery.dequeue( this, type ); + } + } ); + }, + finish: function( type ) { + if ( type !== false ) { + type = type || "fx"; + } + return this.each( function() { + var index, + data = dataPriv.get( this ), + queue = data[ type + "queue" ], + hooks = data[ type + "queueHooks" ], + timers = jQuery.timers, + length = queue ? queue.length : 0; + + // Enable finishing flag on private data + data.finish = true; + + // Empty the queue first + jQuery.queue( this, type, [] ); + + if ( hooks && hooks.stop ) { + hooks.stop.call( this, true ); + } + + // Look for any active animations, and finish them + for ( index = timers.length; index--; ) { + if ( timers[ index ].elem === this && timers[ index ].queue === type ) { + timers[ index ].anim.stop( true ); + timers.splice( index, 1 ); + } + } + + // Look for any animations in the old queue and finish them + for ( index = 0; index < length; index++ ) { + if ( queue[ index ] && queue[ index ].finish ) { + queue[ index ].finish.call( this ); + } + } + + // Turn off finishing flag + delete data.finish; + } ); + } +} ); + +jQuery.each( [ "toggle", "show", "hide" ], function( _i, name ) { + var cssFn = jQuery.fn[ name ]; + jQuery.fn[ name ] = function( speed, easing, callback ) { + return speed == null || typeof speed === "boolean" ? + cssFn.apply( this, arguments ) : + this.animate( genFx( name, true ), speed, easing, callback ); + }; +} ); + +// Generate shortcuts for custom animations +jQuery.each( { + slideDown: genFx( "show" ), + slideUp: genFx( "hide" ), + slideToggle: genFx( "toggle" ), + fadeIn: { opacity: "show" }, + fadeOut: { opacity: "hide" }, + fadeToggle: { opacity: "toggle" } +}, function( name, props ) { + jQuery.fn[ name ] = function( speed, easing, callback ) { + return this.animate( props, speed, easing, callback ); + }; +} ); + +jQuery.timers = []; +jQuery.fx.tick = function() { + var timer, + i = 0, + timers = jQuery.timers; + + fxNow = Date.now(); + + for ( ; i < timers.length; i++ ) { + timer = timers[ i ]; + + // Run the timer and safely remove it when done (allowing for external removal) + if ( !timer() && timers[ i ] === timer ) { + timers.splice( i--, 1 ); + } + } + + if ( !timers.length ) { + jQuery.fx.stop(); + } + fxNow = undefined; +}; + +jQuery.fx.timer = function( timer ) { + jQuery.timers.push( timer ); + jQuery.fx.start(); +}; + +jQuery.fx.interval = 13; +jQuery.fx.start = function() { + if ( inProgress ) { + return; + } + + inProgress = true; + schedule(); +}; + +jQuery.fx.stop = function() { + inProgress = null; +}; + +jQuery.fx.speeds = { + slow: 600, + fast: 200, + + // Default speed + _default: 400 +}; + + +// Based off of the plugin by Clint Helfers, with permission. +// https://web.archive.org/web/20100324014747/http://blindsignals.com/index.php/2009/07/jquery-delay/ +jQuery.fn.delay = function( time, type ) { + time = jQuery.fx ? jQuery.fx.speeds[ time ] || time : time; + type = type || "fx"; + + return this.queue( type, function( next, hooks ) { + var timeout = window.setTimeout( next, time ); + hooks.stop = function() { + window.clearTimeout( timeout ); + }; + } ); +}; + + +( function() { + var input = document.createElement( "input" ), + select = document.createElement( "select" ), + opt = select.appendChild( document.createElement( "option" ) ); + + input.type = "checkbox"; + + // Support: Android <=4.3 only + // Default value for a checkbox should be "on" + support.checkOn = input.value !== ""; + + // Support: IE <=11 only + // Must access selectedIndex to make default options select + support.optSelected = opt.selected; + + // Support: IE <=11 only + // An input loses its value after becoming a radio + input = document.createElement( "input" ); + input.value = "t"; + input.type = "radio"; + support.radioValue = input.value === "t"; +} )(); + + +var boolHook, + attrHandle = jQuery.expr.attrHandle; + +jQuery.fn.extend( { + attr: function( name, value ) { + return access( this, jQuery.attr, name, value, arguments.length > 1 ); + }, + + removeAttr: function( name ) { + return this.each( function() { + jQuery.removeAttr( this, name ); + } ); + } +} ); + +jQuery.extend( { + attr: function( elem, name, value ) { + var ret, hooks, + nType = elem.nodeType; + + // Don't get/set attributes on text, comment and attribute nodes + if ( nType === 3 || nType === 8 || nType === 2 ) { + return; + } + + // Fallback to prop when attributes are not supported + if ( typeof elem.getAttribute === "undefined" ) { + return jQuery.prop( elem, name, value ); + } + + // Attribute hooks are determined by the lowercase version + // Grab necessary hook if one is defined + if ( nType !== 1 || !jQuery.isXMLDoc( elem ) ) { + hooks = jQuery.attrHooks[ name.toLowerCase() ] || + ( jQuery.expr.match.bool.test( name ) ? boolHook : undefined ); + } + + if ( value !== undefined ) { + if ( value === null ) { + jQuery.removeAttr( elem, name ); + return; + } + + if ( hooks && "set" in hooks && + ( ret = hooks.set( elem, value, name ) ) !== undefined ) { + return ret; + } + + elem.setAttribute( name, value + "" ); + return value; + } + + if ( hooks && "get" in hooks && ( ret = hooks.get( elem, name ) ) !== null ) { + return ret; + } + + ret = jQuery.find.attr( elem, name ); + + // Non-existent attributes return null, we normalize to undefined + return ret == null ? undefined : ret; + }, + + attrHooks: { + type: { + set: function( elem, value ) { + if ( !support.radioValue && value === "radio" && + nodeName( elem, "input" ) ) { + var val = elem.value; + elem.setAttribute( "type", value ); + if ( val ) { + elem.value = val; + } + return value; + } + } + } + }, + + removeAttr: function( elem, value ) { + var name, + i = 0, + + // Attribute names can contain non-HTML whitespace characters + // https://html.spec.whatwg.org/multipage/syntax.html#attributes-2 + attrNames = value && value.match( rnothtmlwhite ); + + if ( attrNames && elem.nodeType === 1 ) { + while ( ( name = attrNames[ i++ ] ) ) { + elem.removeAttribute( name ); + } + } + } +} ); + +// Hooks for boolean attributes +boolHook = { + set: function( elem, value, name ) { + if ( value === false ) { + + // Remove boolean attributes when set to false + jQuery.removeAttr( elem, name ); + } else { + elem.setAttribute( name, name ); + } + return name; + } +}; + +jQuery.each( jQuery.expr.match.bool.source.match( /\w+/g ), function( _i, name ) { + var getter = attrHandle[ name ] || jQuery.find.attr; + + attrHandle[ name ] = function( elem, name, isXML ) { + var ret, handle, + lowercaseName = name.toLowerCase(); + + if ( !isXML ) { + + // Avoid an infinite loop by temporarily removing this function from the getter + handle = attrHandle[ lowercaseName ]; + attrHandle[ lowercaseName ] = ret; + ret = getter( elem, name, isXML ) != null ? + lowercaseName : + null; + attrHandle[ lowercaseName ] = handle; + } + return ret; + }; +} ); + + + + +var rfocusable = /^(?:input|select|textarea|button)$/i, + rclickable = /^(?:a|area)$/i; + +jQuery.fn.extend( { + prop: function( name, value ) { + return access( this, jQuery.prop, name, value, arguments.length > 1 ); + }, + + removeProp: function( name ) { + return this.each( function() { + delete this[ jQuery.propFix[ name ] || name ]; + } ); + } +} ); + +jQuery.extend( { + prop: function( elem, name, value ) { + var ret, hooks, + nType = elem.nodeType; + + // Don't get/set properties on text, comment and attribute nodes + if ( nType === 3 || nType === 8 || nType === 2 ) { + return; + } + + if ( nType !== 1 || !jQuery.isXMLDoc( elem ) ) { + + // Fix name and attach hooks + name = jQuery.propFix[ name ] || name; + hooks = jQuery.propHooks[ name ]; + } + + if ( value !== undefined ) { + if ( hooks && "set" in hooks && + ( ret = hooks.set( elem, value, name ) ) !== undefined ) { + return ret; + } + + return ( elem[ name ] = value ); + } + + if ( hooks && "get" in hooks && ( ret = hooks.get( elem, name ) ) !== null ) { + return ret; + } + + return elem[ name ]; + }, + + propHooks: { + tabIndex: { + get: function( elem ) { + + // Support: IE <=9 - 11 only + // elem.tabIndex doesn't always return the + // correct value when it hasn't been explicitly set + // https://web.archive.org/web/20141116233347/http://fluidproject.org/blog/2008/01/09/getting-setting-and-removing-tabindex-values-with-javascript/ + // Use proper attribute retrieval(#12072) + var tabindex = jQuery.find.attr( elem, "tabindex" ); + + if ( tabindex ) { + return parseInt( tabindex, 10 ); + } + + if ( + rfocusable.test( elem.nodeName ) || + rclickable.test( elem.nodeName ) && + elem.href + ) { + return 0; + } + + return -1; + } + } + }, + + propFix: { + "for": "htmlFor", + "class": "className" + } +} ); + +// Support: IE <=11 only +// Accessing the selectedIndex property +// forces the browser to respect setting selected +// on the option +// The getter ensures a default option is selected +// when in an optgroup +// eslint rule "no-unused-expressions" is disabled for this code +// since it considers such accessions noop +if ( !support.optSelected ) { + jQuery.propHooks.selected = { + get: function( elem ) { + + /* eslint no-unused-expressions: "off" */ + + var parent = elem.parentNode; + if ( parent && parent.parentNode ) { + parent.parentNode.selectedIndex; + } + return null; + }, + set: function( elem ) { + + /* eslint no-unused-expressions: "off" */ + + var parent = elem.parentNode; + if ( parent ) { + parent.selectedIndex; + + if ( parent.parentNode ) { + parent.parentNode.selectedIndex; + } + } + } + }; +} + +jQuery.each( [ + "tabIndex", + "readOnly", + "maxLength", + "cellSpacing", + "cellPadding", + "rowSpan", + "colSpan", + "useMap", + "frameBorder", + "contentEditable" +], function() { + jQuery.propFix[ this.toLowerCase() ] = this; +} ); + + + + + // Strip and collapse whitespace according to HTML spec + // https://infra.spec.whatwg.org/#strip-and-collapse-ascii-whitespace + function stripAndCollapse( value ) { + var tokens = value.match( rnothtmlwhite ) || []; + return tokens.join( " " ); + } + + +function getClass( elem ) { + return elem.getAttribute && elem.getAttribute( "class" ) || ""; +} + +function classesToArray( value ) { + if ( Array.isArray( value ) ) { + return value; + } + if ( typeof value === "string" ) { + return value.match( rnothtmlwhite ) || []; + } + return []; +} + +jQuery.fn.extend( { + addClass: function( value ) { + var classes, elem, cur, curValue, clazz, j, finalValue, + i = 0; + + if ( isFunction( value ) ) { + return this.each( function( j ) { + jQuery( this ).addClass( value.call( this, j, getClass( this ) ) ); + } ); + } + + classes = classesToArray( value ); + + if ( classes.length ) { + while ( ( elem = this[ i++ ] ) ) { + curValue = getClass( elem ); + cur = elem.nodeType === 1 && ( " " + stripAndCollapse( curValue ) + " " ); + + if ( cur ) { + j = 0; + while ( ( clazz = classes[ j++ ] ) ) { + if ( cur.indexOf( " " + clazz + " " ) < 0 ) { + cur += clazz + " "; + } + } + + // Only assign if different to avoid unneeded rendering. + finalValue = stripAndCollapse( cur ); + if ( curValue !== finalValue ) { + elem.setAttribute( "class", finalValue ); + } + } + } + } + + return this; + }, + + removeClass: function( value ) { + var classes, elem, cur, curValue, clazz, j, finalValue, + i = 0; + + if ( isFunction( value ) ) { + return this.each( function( j ) { + jQuery( this ).removeClass( value.call( this, j, getClass( this ) ) ); + } ); + } + + if ( !arguments.length ) { + return this.attr( "class", "" ); + } + + classes = classesToArray( value ); + + if ( classes.length ) { + while ( ( elem = this[ i++ ] ) ) { + curValue = getClass( elem ); + + // This expression is here for better compressibility (see addClass) + cur = elem.nodeType === 1 && ( " " + stripAndCollapse( curValue ) + " " ); + + if ( cur ) { + j = 0; + while ( ( clazz = classes[ j++ ] ) ) { + + // Remove *all* instances + while ( cur.indexOf( " " + clazz + " " ) > -1 ) { + cur = cur.replace( " " + clazz + " ", " " ); + } + } + + // Only assign if different to avoid unneeded rendering. + finalValue = stripAndCollapse( cur ); + if ( curValue !== finalValue ) { + elem.setAttribute( "class", finalValue ); + } + } + } + } + + return this; + }, + + toggleClass: function( value, stateVal ) { + var type = typeof value, + isValidValue = type === "string" || Array.isArray( value ); + + if ( typeof stateVal === "boolean" && isValidValue ) { + return stateVal ? this.addClass( value ) : this.removeClass( value ); + } + + if ( isFunction( value ) ) { + return this.each( function( i ) { + jQuery( this ).toggleClass( + value.call( this, i, getClass( this ), stateVal ), + stateVal + ); + } ); + } + + return this.each( function() { + var className, i, self, classNames; + + if ( isValidValue ) { + + // Toggle individual class names + i = 0; + self = jQuery( this ); + classNames = classesToArray( value ); + + while ( ( className = classNames[ i++ ] ) ) { + + // Check each className given, space separated list + if ( self.hasClass( className ) ) { + self.removeClass( className ); + } else { + self.addClass( className ); + } + } + + // Toggle whole class name + } else if ( value === undefined || type === "boolean" ) { + className = getClass( this ); + if ( className ) { + + // Store className if set + dataPriv.set( this, "__className__", className ); + } + + // If the element has a class name or if we're passed `false`, + // then remove the whole classname (if there was one, the above saved it). + // Otherwise bring back whatever was previously saved (if anything), + // falling back to the empty string if nothing was stored. + if ( this.setAttribute ) { + this.setAttribute( "class", + className || value === false ? + "" : + dataPriv.get( this, "__className__" ) || "" + ); + } + } + } ); + }, + + hasClass: function( selector ) { + var className, elem, + i = 0; + + className = " " + selector + " "; + while ( ( elem = this[ i++ ] ) ) { + if ( elem.nodeType === 1 && + ( " " + stripAndCollapse( getClass( elem ) ) + " " ).indexOf( className ) > -1 ) { + return true; + } + } + + return false; + } +} ); + + + + +var rreturn = /\r/g; + +jQuery.fn.extend( { + val: function( value ) { + var hooks, ret, valueIsFunction, + elem = this[ 0 ]; + + if ( !arguments.length ) { + if ( elem ) { + hooks = jQuery.valHooks[ elem.type ] || + jQuery.valHooks[ elem.nodeName.toLowerCase() ]; + + if ( hooks && + "get" in hooks && + ( ret = hooks.get( elem, "value" ) ) !== undefined + ) { + return ret; + } + + ret = elem.value; + + // Handle most common string cases + if ( typeof ret === "string" ) { + return ret.replace( rreturn, "" ); + } + + // Handle cases where value is null/undef or number + return ret == null ? "" : ret; + } + + return; + } + + valueIsFunction = isFunction( value ); + + return this.each( function( i ) { + var val; + + if ( this.nodeType !== 1 ) { + return; + } + + if ( valueIsFunction ) { + val = value.call( this, i, jQuery( this ).val() ); + } else { + val = value; + } + + // Treat null/undefined as ""; convert numbers to string + if ( val == null ) { + val = ""; + + } else if ( typeof val === "number" ) { + val += ""; + + } else if ( Array.isArray( val ) ) { + val = jQuery.map( val, function( value ) { + return value == null ? "" : value + ""; + } ); + } + + hooks = jQuery.valHooks[ this.type ] || jQuery.valHooks[ this.nodeName.toLowerCase() ]; + + // If set returns undefined, fall back to normal setting + if ( !hooks || !( "set" in hooks ) || hooks.set( this, val, "value" ) === undefined ) { + this.value = val; + } + } ); + } +} ); + +jQuery.extend( { + valHooks: { + option: { + get: function( elem ) { + + var val = jQuery.find.attr( elem, "value" ); + return val != null ? + val : + + // Support: IE <=10 - 11 only + // option.text throws exceptions (#14686, #14858) + // Strip and collapse whitespace + // https://html.spec.whatwg.org/#strip-and-collapse-whitespace + stripAndCollapse( jQuery.text( elem ) ); + } + }, + select: { + get: function( elem ) { + var value, option, i, + options = elem.options, + index = elem.selectedIndex, + one = elem.type === "select-one", + values = one ? null : [], + max = one ? index + 1 : options.length; + + if ( index < 0 ) { + i = max; + + } else { + i = one ? index : 0; + } + + // Loop through all the selected options + for ( ; i < max; i++ ) { + option = options[ i ]; + + // Support: IE <=9 only + // IE8-9 doesn't update selected after form reset (#2551) + if ( ( option.selected || i === index ) && + + // Don't return options that are disabled or in a disabled optgroup + !option.disabled && + ( !option.parentNode.disabled || + !nodeName( option.parentNode, "optgroup" ) ) ) { + + // Get the specific value for the option + value = jQuery( option ).val(); + + // We don't need an array for one selects + if ( one ) { + return value; + } + + // Multi-Selects return an array + values.push( value ); + } + } + + return values; + }, + + set: function( elem, value ) { + var optionSet, option, + options = elem.options, + values = jQuery.makeArray( value ), + i = options.length; + + while ( i-- ) { + option = options[ i ]; + + /* eslint-disable no-cond-assign */ + + if ( option.selected = + jQuery.inArray( jQuery.valHooks.option.get( option ), values ) > -1 + ) { + optionSet = true; + } + + /* eslint-enable no-cond-assign */ + } + + // Force browsers to behave consistently when non-matching value is set + if ( !optionSet ) { + elem.selectedIndex = -1; + } + return values; + } + } + } +} ); + +// Radios and checkboxes getter/setter +jQuery.each( [ "radio", "checkbox" ], function() { + jQuery.valHooks[ this ] = { + set: function( elem, value ) { + if ( Array.isArray( value ) ) { + return ( elem.checked = jQuery.inArray( jQuery( elem ).val(), value ) > -1 ); + } + } + }; + if ( !support.checkOn ) { + jQuery.valHooks[ this ].get = function( elem ) { + return elem.getAttribute( "value" ) === null ? "on" : elem.value; + }; + } +} ); + + + + +// Return jQuery for attributes-only inclusion + + +support.focusin = "onfocusin" in window; + + +var rfocusMorph = /^(?:focusinfocus|focusoutblur)$/, + stopPropagationCallback = function( e ) { + e.stopPropagation(); + }; + +jQuery.extend( jQuery.event, { + + trigger: function( event, data, elem, onlyHandlers ) { + + var i, cur, tmp, bubbleType, ontype, handle, special, lastElement, + eventPath = [ elem || document ], + type = hasOwn.call( event, "type" ) ? event.type : event, + namespaces = hasOwn.call( event, "namespace" ) ? event.namespace.split( "." ) : []; + + cur = lastElement = tmp = elem = elem || document; + + // Don't do events on text and comment nodes + if ( elem.nodeType === 3 || elem.nodeType === 8 ) { + return; + } + + // focus/blur morphs to focusin/out; ensure we're not firing them right now + if ( rfocusMorph.test( type + jQuery.event.triggered ) ) { + return; + } + + if ( type.indexOf( "." ) > -1 ) { + + // Namespaced trigger; create a regexp to match event type in handle() + namespaces = type.split( "." ); + type = namespaces.shift(); + namespaces.sort(); + } + ontype = type.indexOf( ":" ) < 0 && "on" + type; + + // Caller can pass in a jQuery.Event object, Object, or just an event type string + event = event[ jQuery.expando ] ? + event : + new jQuery.Event( type, typeof event === "object" && event ); + + // Trigger bitmask: & 1 for native handlers; & 2 for jQuery (always true) + event.isTrigger = onlyHandlers ? 2 : 3; + event.namespace = namespaces.join( "." ); + event.rnamespace = event.namespace ? + new RegExp( "(^|\\.)" + namespaces.join( "\\.(?:.*\\.|)" ) + "(\\.|$)" ) : + null; + + // Clean up the event in case it is being reused + event.result = undefined; + if ( !event.target ) { + event.target = elem; + } + + // Clone any incoming data and prepend the event, creating the handler arg list + data = data == null ? + [ event ] : + jQuery.makeArray( data, [ event ] ); + + // Allow special events to draw outside the lines + special = jQuery.event.special[ type ] || {}; + if ( !onlyHandlers && special.trigger && special.trigger.apply( elem, data ) === false ) { + return; + } + + // Determine event propagation path in advance, per W3C events spec (#9951) + // Bubble up to document, then to window; watch for a global ownerDocument var (#9724) + if ( !onlyHandlers && !special.noBubble && !isWindow( elem ) ) { + + bubbleType = special.delegateType || type; + if ( !rfocusMorph.test( bubbleType + type ) ) { + cur = cur.parentNode; + } + for ( ; cur; cur = cur.parentNode ) { + eventPath.push( cur ); + tmp = cur; + } + + // Only add window if we got to document (e.g., not plain obj or detached DOM) + if ( tmp === ( elem.ownerDocument || document ) ) { + eventPath.push( tmp.defaultView || tmp.parentWindow || window ); + } + } + + // Fire handlers on the event path + i = 0; + while ( ( cur = eventPath[ i++ ] ) && !event.isPropagationStopped() ) { + lastElement = cur; + event.type = i > 1 ? + bubbleType : + special.bindType || type; + + // jQuery handler + handle = ( dataPriv.get( cur, "events" ) || Object.create( null ) )[ event.type ] && + dataPriv.get( cur, "handle" ); + if ( handle ) { + handle.apply( cur, data ); + } + + // Native handler + handle = ontype && cur[ ontype ]; + if ( handle && handle.apply && acceptData( cur ) ) { + event.result = handle.apply( cur, data ); + if ( event.result === false ) { + event.preventDefault(); + } + } + } + event.type = type; + + // If nobody prevented the default action, do it now + if ( !onlyHandlers && !event.isDefaultPrevented() ) { + + if ( ( !special._default || + special._default.apply( eventPath.pop(), data ) === false ) && + acceptData( elem ) ) { + + // Call a native DOM method on the target with the same name as the event. + // Don't do default actions on window, that's where global variables be (#6170) + if ( ontype && isFunction( elem[ type ] ) && !isWindow( elem ) ) { + + // Don't re-trigger an onFOO event when we call its FOO() method + tmp = elem[ ontype ]; + + if ( tmp ) { + elem[ ontype ] = null; + } + + // Prevent re-triggering of the same event, since we already bubbled it above + jQuery.event.triggered = type; + + if ( event.isPropagationStopped() ) { + lastElement.addEventListener( type, stopPropagationCallback ); + } + + elem[ type ](); + + if ( event.isPropagationStopped() ) { + lastElement.removeEventListener( type, stopPropagationCallback ); + } + + jQuery.event.triggered = undefined; + + if ( tmp ) { + elem[ ontype ] = tmp; + } + } + } + } + + return event.result; + }, + + // Piggyback on a donor event to simulate a different one + // Used only for `focus(in | out)` events + simulate: function( type, elem, event ) { + var e = jQuery.extend( + new jQuery.Event(), + event, + { + type: type, + isSimulated: true + } + ); + + jQuery.event.trigger( e, null, elem ); + } + +} ); + +jQuery.fn.extend( { + + trigger: function( type, data ) { + return this.each( function() { + jQuery.event.trigger( type, data, this ); + } ); + }, + triggerHandler: function( type, data ) { + var elem = this[ 0 ]; + if ( elem ) { + return jQuery.event.trigger( type, data, elem, true ); + } + } +} ); + + +// Support: Firefox <=44 +// Firefox doesn't have focus(in | out) events +// Related ticket - https://bugzilla.mozilla.org/show_bug.cgi?id=687787 +// +// Support: Chrome <=48 - 49, Safari <=9.0 - 9.1 +// focus(in | out) events fire after focus & blur events, +// which is spec violation - http://www.w3.org/TR/DOM-Level-3-Events/#events-focusevent-event-order +// Related ticket - https://bugs.chromium.org/p/chromium/issues/detail?id=449857 +if ( !support.focusin ) { + jQuery.each( { focus: "focusin", blur: "focusout" }, function( orig, fix ) { + + // Attach a single capturing handler on the document while someone wants focusin/focusout + var handler = function( event ) { + jQuery.event.simulate( fix, event.target, jQuery.event.fix( event ) ); + }; + + jQuery.event.special[ fix ] = { + setup: function() { + + // Handle: regular nodes (via `this.ownerDocument`), window + // (via `this.document`) & document (via `this`). + var doc = this.ownerDocument || this.document || this, + attaches = dataPriv.access( doc, fix ); + + if ( !attaches ) { + doc.addEventListener( orig, handler, true ); + } + dataPriv.access( doc, fix, ( attaches || 0 ) + 1 ); + }, + teardown: function() { + var doc = this.ownerDocument || this.document || this, + attaches = dataPriv.access( doc, fix ) - 1; + + if ( !attaches ) { + doc.removeEventListener( orig, handler, true ); + dataPriv.remove( doc, fix ); + + } else { + dataPriv.access( doc, fix, attaches ); + } + } + }; + } ); +} +var location = window.location; + +var nonce = { guid: Date.now() }; + +var rquery = ( /\?/ ); + + + +// Cross-browser xml parsing +jQuery.parseXML = function( data ) { + var xml, parserErrorElem; + if ( !data || typeof data !== "string" ) { + return null; + } + + // Support: IE 9 - 11 only + // IE throws on parseFromString with invalid input. + try { + xml = ( new window.DOMParser() ).parseFromString( data, "text/xml" ); + } catch ( e ) {} + + parserErrorElem = xml && xml.getElementsByTagName( "parsererror" )[ 0 ]; + if ( !xml || parserErrorElem ) { + jQuery.error( "Invalid XML: " + ( + parserErrorElem ? + jQuery.map( parserErrorElem.childNodes, function( el ) { + return el.textContent; + } ).join( "\n" ) : + data + ) ); + } + return xml; +}; + + +var + rbracket = /\[\]$/, + rCRLF = /\r?\n/g, + rsubmitterTypes = /^(?:submit|button|image|reset|file)$/i, + rsubmittable = /^(?:input|select|textarea|keygen)/i; + +function buildParams( prefix, obj, traditional, add ) { + var name; + + if ( Array.isArray( obj ) ) { + + // Serialize array item. + jQuery.each( obj, function( i, v ) { + if ( traditional || rbracket.test( prefix ) ) { + + // Treat each array item as a scalar. + add( prefix, v ); + + } else { + + // Item is non-scalar (array or object), encode its numeric index. + buildParams( + prefix + "[" + ( typeof v === "object" && v != null ? i : "" ) + "]", + v, + traditional, + add + ); + } + } ); + + } else if ( !traditional && toType( obj ) === "object" ) { + + // Serialize object item. + for ( name in obj ) { + buildParams( prefix + "[" + name + "]", obj[ name ], traditional, add ); + } + + } else { + + // Serialize scalar item. + add( prefix, obj ); + } +} + +// Serialize an array of form elements or a set of +// key/values into a query string +jQuery.param = function( a, traditional ) { + var prefix, + s = [], + add = function( key, valueOrFunction ) { + + // If value is a function, invoke it and use its return value + var value = isFunction( valueOrFunction ) ? + valueOrFunction() : + valueOrFunction; + + s[ s.length ] = encodeURIComponent( key ) + "=" + + encodeURIComponent( value == null ? "" : value ); + }; + + if ( a == null ) { + return ""; + } + + // If an array was passed in, assume that it is an array of form elements. + if ( Array.isArray( a ) || ( a.jquery && !jQuery.isPlainObject( a ) ) ) { + + // Serialize the form elements + jQuery.each( a, function() { + add( this.name, this.value ); + } ); + + } else { + + // If traditional, encode the "old" way (the way 1.3.2 or older + // did it), otherwise encode params recursively. + for ( prefix in a ) { + buildParams( prefix, a[ prefix ], traditional, add ); + } + } + + // Return the resulting serialization + return s.join( "&" ); +}; + +jQuery.fn.extend( { + serialize: function() { + return jQuery.param( this.serializeArray() ); + }, + serializeArray: function() { + return this.map( function() { + + // Can add propHook for "elements" to filter or add form elements + var elements = jQuery.prop( this, "elements" ); + return elements ? jQuery.makeArray( elements ) : this; + } ).filter( function() { + var type = this.type; + + // Use .is( ":disabled" ) so that fieldset[disabled] works + return this.name && !jQuery( this ).is( ":disabled" ) && + rsubmittable.test( this.nodeName ) && !rsubmitterTypes.test( type ) && + ( this.checked || !rcheckableType.test( type ) ); + } ).map( function( _i, elem ) { + var val = jQuery( this ).val(); + + if ( val == null ) { + return null; + } + + if ( Array.isArray( val ) ) { + return jQuery.map( val, function( val ) { + return { name: elem.name, value: val.replace( rCRLF, "\r\n" ) }; + } ); + } + + return { name: elem.name, value: val.replace( rCRLF, "\r\n" ) }; + } ).get(); + } +} ); + + +var + r20 = /%20/g, + rhash = /#.*$/, + rantiCache = /([?&])_=[^&]*/, + rheaders = /^(.*?):[ \t]*([^\r\n]*)$/mg, + + // #7653, #8125, #8152: local protocol detection + rlocalProtocol = /^(?:about|app|app-storage|.+-extension|file|res|widget):$/, + rnoContent = /^(?:GET|HEAD)$/, + rprotocol = /^\/\//, + + /* Prefilters + * 1) They are useful to introduce custom dataTypes (see ajax/jsonp.js for an example) + * 2) These are called: + * - BEFORE asking for a transport + * - AFTER param serialization (s.data is a string if s.processData is true) + * 3) key is the dataType + * 4) the catchall symbol "*" can be used + * 5) execution will start with transport dataType and THEN continue down to "*" if needed + */ + prefilters = {}, + + /* Transports bindings + * 1) key is the dataType + * 2) the catchall symbol "*" can be used + * 3) selection will start with transport dataType and THEN go to "*" if needed + */ + transports = {}, + + // Avoid comment-prolog char sequence (#10098); must appease lint and evade compression + allTypes = "*/".concat( "*" ), + + // Anchor tag for parsing the document origin + originAnchor = document.createElement( "a" ); + +originAnchor.href = location.href; + +// Base "constructor" for jQuery.ajaxPrefilter and jQuery.ajaxTransport +function addToPrefiltersOrTransports( structure ) { + + // dataTypeExpression is optional and defaults to "*" + return function( dataTypeExpression, func ) { + + if ( typeof dataTypeExpression !== "string" ) { + func = dataTypeExpression; + dataTypeExpression = "*"; + } + + var dataType, + i = 0, + dataTypes = dataTypeExpression.toLowerCase().match( rnothtmlwhite ) || []; + + if ( isFunction( func ) ) { + + // For each dataType in the dataTypeExpression + while ( ( dataType = dataTypes[ i++ ] ) ) { + + // Prepend if requested + if ( dataType[ 0 ] === "+" ) { + dataType = dataType.slice( 1 ) || "*"; + ( structure[ dataType ] = structure[ dataType ] || [] ).unshift( func ); + + // Otherwise append + } else { + ( structure[ dataType ] = structure[ dataType ] || [] ).push( func ); + } + } + } + }; +} + +// Base inspection function for prefilters and transports +function inspectPrefiltersOrTransports( structure, options, originalOptions, jqXHR ) { + + var inspected = {}, + seekingTransport = ( structure === transports ); + + function inspect( dataType ) { + var selected; + inspected[ dataType ] = true; + jQuery.each( structure[ dataType ] || [], function( _, prefilterOrFactory ) { + var dataTypeOrTransport = prefilterOrFactory( options, originalOptions, jqXHR ); + if ( typeof dataTypeOrTransport === "string" && + !seekingTransport && !inspected[ dataTypeOrTransport ] ) { + + options.dataTypes.unshift( dataTypeOrTransport ); + inspect( dataTypeOrTransport ); + return false; + } else if ( seekingTransport ) { + return !( selected = dataTypeOrTransport ); + } + } ); + return selected; + } + + return inspect( options.dataTypes[ 0 ] ) || !inspected[ "*" ] && inspect( "*" ); +} + +// A special extend for ajax options +// that takes "flat" options (not to be deep extended) +// Fixes #9887 +function ajaxExtend( target, src ) { + var key, deep, + flatOptions = jQuery.ajaxSettings.flatOptions || {}; + + for ( key in src ) { + if ( src[ key ] !== undefined ) { + ( flatOptions[ key ] ? target : ( deep || ( deep = {} ) ) )[ key ] = src[ key ]; + } + } + if ( deep ) { + jQuery.extend( true, target, deep ); + } + + return target; +} + +/* Handles responses to an ajax request: + * - finds the right dataType (mediates between content-type and expected dataType) + * - returns the corresponding response + */ +function ajaxHandleResponses( s, jqXHR, responses ) { + + var ct, type, finalDataType, firstDataType, + contents = s.contents, + dataTypes = s.dataTypes; + + // Remove auto dataType and get content-type in the process + while ( dataTypes[ 0 ] === "*" ) { + dataTypes.shift(); + if ( ct === undefined ) { + ct = s.mimeType || jqXHR.getResponseHeader( "Content-Type" ); + } + } + + // Check if we're dealing with a known content-type + if ( ct ) { + for ( type in contents ) { + if ( contents[ type ] && contents[ type ].test( ct ) ) { + dataTypes.unshift( type ); + break; + } + } + } + + // Check to see if we have a response for the expected dataType + if ( dataTypes[ 0 ] in responses ) { + finalDataType = dataTypes[ 0 ]; + } else { + + // Try convertible dataTypes + for ( type in responses ) { + if ( !dataTypes[ 0 ] || s.converters[ type + " " + dataTypes[ 0 ] ] ) { + finalDataType = type; + break; + } + if ( !firstDataType ) { + firstDataType = type; + } + } + + // Or just use first one + finalDataType = finalDataType || firstDataType; + } + + // If we found a dataType + // We add the dataType to the list if needed + // and return the corresponding response + if ( finalDataType ) { + if ( finalDataType !== dataTypes[ 0 ] ) { + dataTypes.unshift( finalDataType ); + } + return responses[ finalDataType ]; + } +} + +/* Chain conversions given the request and the original response + * Also sets the responseXXX fields on the jqXHR instance + */ +function ajaxConvert( s, response, jqXHR, isSuccess ) { + var conv2, current, conv, tmp, prev, + converters = {}, + + // Work with a copy of dataTypes in case we need to modify it for conversion + dataTypes = s.dataTypes.slice(); + + // Create converters map with lowercased keys + if ( dataTypes[ 1 ] ) { + for ( conv in s.converters ) { + converters[ conv.toLowerCase() ] = s.converters[ conv ]; + } + } + + current = dataTypes.shift(); + + // Convert to each sequential dataType + while ( current ) { + + if ( s.responseFields[ current ] ) { + jqXHR[ s.responseFields[ current ] ] = response; + } + + // Apply the dataFilter if provided + if ( !prev && isSuccess && s.dataFilter ) { + response = s.dataFilter( response, s.dataType ); + } + + prev = current; + current = dataTypes.shift(); + + if ( current ) { + + // There's only work to do if current dataType is non-auto + if ( current === "*" ) { + + current = prev; + + // Convert response if prev dataType is non-auto and differs from current + } else if ( prev !== "*" && prev !== current ) { + + // Seek a direct converter + conv = converters[ prev + " " + current ] || converters[ "* " + current ]; + + // If none found, seek a pair + if ( !conv ) { + for ( conv2 in converters ) { + + // If conv2 outputs current + tmp = conv2.split( " " ); + if ( tmp[ 1 ] === current ) { + + // If prev can be converted to accepted input + conv = converters[ prev + " " + tmp[ 0 ] ] || + converters[ "* " + tmp[ 0 ] ]; + if ( conv ) { + + // Condense equivalence converters + if ( conv === true ) { + conv = converters[ conv2 ]; + + // Otherwise, insert the intermediate dataType + } else if ( converters[ conv2 ] !== true ) { + current = tmp[ 0 ]; + dataTypes.unshift( tmp[ 1 ] ); + } + break; + } + } + } + } + + // Apply converter (if not an equivalence) + if ( conv !== true ) { + + // Unless errors are allowed to bubble, catch and return them + if ( conv && s.throws ) { + response = conv( response ); + } else { + try { + response = conv( response ); + } catch ( e ) { + return { + state: "parsererror", + error: conv ? e : "No conversion from " + prev + " to " + current + }; + } + } + } + } + } + } + + return { state: "success", data: response }; +} + +jQuery.extend( { + + // Counter for holding the number of active queries + active: 0, + + // Last-Modified header cache for next request + lastModified: {}, + etag: {}, + + ajaxSettings: { + url: location.href, + type: "GET", + isLocal: rlocalProtocol.test( location.protocol ), + global: true, + processData: true, + async: true, + contentType: "application/x-www-form-urlencoded; charset=UTF-8", + + /* + timeout: 0, + data: null, + dataType: null, + username: null, + password: null, + cache: null, + throws: false, + traditional: false, + headers: {}, + */ + + accepts: { + "*": allTypes, + text: "text/plain", + html: "text/html", + xml: "application/xml, text/xml", + json: "application/json, text/javascript" + }, + + contents: { + xml: /\bxml\b/, + html: /\bhtml/, + json: /\bjson\b/ + }, + + responseFields: { + xml: "responseXML", + text: "responseText", + json: "responseJSON" + }, + + // Data converters + // Keys separate source (or catchall "*") and destination types with a single space + converters: { + + // Convert anything to text + "* text": String, + + // Text to html (true = no transformation) + "text html": true, + + // Evaluate text as a json expression + "text json": JSON.parse, + + // Parse text as xml + "text xml": jQuery.parseXML + }, + + // For options that shouldn't be deep extended: + // you can add your own custom options here if + // and when you create one that shouldn't be + // deep extended (see ajaxExtend) + flatOptions: { + url: true, + context: true + } + }, + + // Creates a full fledged settings object into target + // with both ajaxSettings and settings fields. + // If target is omitted, writes into ajaxSettings. + ajaxSetup: function( target, settings ) { + return settings ? + + // Building a settings object + ajaxExtend( ajaxExtend( target, jQuery.ajaxSettings ), settings ) : + + // Extending ajaxSettings + ajaxExtend( jQuery.ajaxSettings, target ); + }, + + ajaxPrefilter: addToPrefiltersOrTransports( prefilters ), + ajaxTransport: addToPrefiltersOrTransports( transports ), + + // Main method + ajax: function( url, options ) { + + // If url is an object, simulate pre-1.5 signature + if ( typeof url === "object" ) { + options = url; + url = undefined; + } + + // Force options to be an object + options = options || {}; + + var transport, + + // URL without anti-cache param + cacheURL, + + // Response headers + responseHeadersString, + responseHeaders, + + // timeout handle + timeoutTimer, + + // Url cleanup var + urlAnchor, + + // Request state (becomes false upon send and true upon completion) + completed, + + // To know if global events are to be dispatched + fireGlobals, + + // Loop variable + i, + + // uncached part of the url + uncached, + + // Create the final options object + s = jQuery.ajaxSetup( {}, options ), + + // Callbacks context + callbackContext = s.context || s, + + // Context for global events is callbackContext if it is a DOM node or jQuery collection + globalEventContext = s.context && + ( callbackContext.nodeType || callbackContext.jquery ) ? + jQuery( callbackContext ) : + jQuery.event, + + // Deferreds + deferred = jQuery.Deferred(), + completeDeferred = jQuery.Callbacks( "once memory" ), + + // Status-dependent callbacks + statusCode = s.statusCode || {}, + + // Headers (they are sent all at once) + requestHeaders = {}, + requestHeadersNames = {}, + + // Default abort message + strAbort = "canceled", + + // Fake xhr + jqXHR = { + readyState: 0, + + // Builds headers hashtable if needed + getResponseHeader: function( key ) { + var match; + if ( completed ) { + if ( !responseHeaders ) { + responseHeaders = {}; + while ( ( match = rheaders.exec( responseHeadersString ) ) ) { + responseHeaders[ match[ 1 ].toLowerCase() + " " ] = + ( responseHeaders[ match[ 1 ].toLowerCase() + " " ] || [] ) + .concat( match[ 2 ] ); + } + } + match = responseHeaders[ key.toLowerCase() + " " ]; + } + return match == null ? null : match.join( ", " ); + }, + + // Raw string + getAllResponseHeaders: function() { + return completed ? responseHeadersString : null; + }, + + // Caches the header + setRequestHeader: function( name, value ) { + if ( completed == null ) { + name = requestHeadersNames[ name.toLowerCase() ] = + requestHeadersNames[ name.toLowerCase() ] || name; + requestHeaders[ name ] = value; + } + return this; + }, + + // Overrides response content-type header + overrideMimeType: function( type ) { + if ( completed == null ) { + s.mimeType = type; + } + return this; + }, + + // Status-dependent callbacks + statusCode: function( map ) { + var code; + if ( map ) { + if ( completed ) { + + // Execute the appropriate callbacks + jqXHR.always( map[ jqXHR.status ] ); + } else { + + // Lazy-add the new callbacks in a way that preserves old ones + for ( code in map ) { + statusCode[ code ] = [ statusCode[ code ], map[ code ] ]; + } + } + } + return this; + }, + + // Cancel the request + abort: function( statusText ) { + var finalText = statusText || strAbort; + if ( transport ) { + transport.abort( finalText ); + } + done( 0, finalText ); + return this; + } + }; + + // Attach deferreds + deferred.promise( jqXHR ); + + // Add protocol if not provided (prefilters might expect it) + // Handle falsy url in the settings object (#10093: consistency with old signature) + // We also use the url parameter if available + s.url = ( ( url || s.url || location.href ) + "" ) + .replace( rprotocol, location.protocol + "//" ); + + // Alias method option to type as per ticket #12004 + s.type = options.method || options.type || s.method || s.type; + + // Extract dataTypes list + s.dataTypes = ( s.dataType || "*" ).toLowerCase().match( rnothtmlwhite ) || [ "" ]; + + // A cross-domain request is in order when the origin doesn't match the current origin. + if ( s.crossDomain == null ) { + urlAnchor = document.createElement( "a" ); + + // Support: IE <=8 - 11, Edge 12 - 15 + // IE throws exception on accessing the href property if url is malformed, + // e.g. http://example.com:80x/ + try { + urlAnchor.href = s.url; + + // Support: IE <=8 - 11 only + // Anchor's host property isn't correctly set when s.url is relative + urlAnchor.href = urlAnchor.href; + s.crossDomain = originAnchor.protocol + "//" + originAnchor.host !== + urlAnchor.protocol + "//" + urlAnchor.host; + } catch ( e ) { + + // If there is an error parsing the URL, assume it is crossDomain, + // it can be rejected by the transport if it is invalid + s.crossDomain = true; + } + } + + // Convert data if not already a string + if ( s.data && s.processData && typeof s.data !== "string" ) { + s.data = jQuery.param( s.data, s.traditional ); + } + + // Apply prefilters + inspectPrefiltersOrTransports( prefilters, s, options, jqXHR ); + + // If request was aborted inside a prefilter, stop there + if ( completed ) { + return jqXHR; + } + + // We can fire global events as of now if asked to + // Don't fire events if jQuery.event is undefined in an AMD-usage scenario (#15118) + fireGlobals = jQuery.event && s.global; + + // Watch for a new set of requests + if ( fireGlobals && jQuery.active++ === 0 ) { + jQuery.event.trigger( "ajaxStart" ); + } + + // Uppercase the type + s.type = s.type.toUpperCase(); + + // Determine if request has content + s.hasContent = !rnoContent.test( s.type ); + + // Save the URL in case we're toying with the If-Modified-Since + // and/or If-None-Match header later on + // Remove hash to simplify url manipulation + cacheURL = s.url.replace( rhash, "" ); + + // More options handling for requests with no content + if ( !s.hasContent ) { + + // Remember the hash so we can put it back + uncached = s.url.slice( cacheURL.length ); + + // If data is available and should be processed, append data to url + if ( s.data && ( s.processData || typeof s.data === "string" ) ) { + cacheURL += ( rquery.test( cacheURL ) ? "&" : "?" ) + s.data; + + // #9682: remove data so that it's not used in an eventual retry + delete s.data; + } + + // Add or update anti-cache param if needed + if ( s.cache === false ) { + cacheURL = cacheURL.replace( rantiCache, "$1" ); + uncached = ( rquery.test( cacheURL ) ? "&" : "?" ) + "_=" + ( nonce.guid++ ) + + uncached; + } + + // Put hash and anti-cache on the URL that will be requested (gh-1732) + s.url = cacheURL + uncached; + + // Change '%20' to '+' if this is encoded form body content (gh-2658) + } else if ( s.data && s.processData && + ( s.contentType || "" ).indexOf( "application/x-www-form-urlencoded" ) === 0 ) { + s.data = s.data.replace( r20, "+" ); + } + + // Set the If-Modified-Since and/or If-None-Match header, if in ifModified mode. + if ( s.ifModified ) { + if ( jQuery.lastModified[ cacheURL ] ) { + jqXHR.setRequestHeader( "If-Modified-Since", jQuery.lastModified[ cacheURL ] ); + } + if ( jQuery.etag[ cacheURL ] ) { + jqXHR.setRequestHeader( "If-None-Match", jQuery.etag[ cacheURL ] ); + } + } + + // Set the correct header, if data is being sent + if ( s.data && s.hasContent && s.contentType !== false || options.contentType ) { + jqXHR.setRequestHeader( "Content-Type", s.contentType ); + } + + // Set the Accepts header for the server, depending on the dataType + jqXHR.setRequestHeader( + "Accept", + s.dataTypes[ 0 ] && s.accepts[ s.dataTypes[ 0 ] ] ? + s.accepts[ s.dataTypes[ 0 ] ] + + ( s.dataTypes[ 0 ] !== "*" ? ", " + allTypes + "; q=0.01" : "" ) : + s.accepts[ "*" ] + ); + + // Check for headers option + for ( i in s.headers ) { + jqXHR.setRequestHeader( i, s.headers[ i ] ); + } + + // Allow custom headers/mimetypes and early abort + if ( s.beforeSend && + ( s.beforeSend.call( callbackContext, jqXHR, s ) === false || completed ) ) { + + // Abort if not done already and return + return jqXHR.abort(); + } + + // Aborting is no longer a cancellation + strAbort = "abort"; + + // Install callbacks on deferreds + completeDeferred.add( s.complete ); + jqXHR.done( s.success ); + jqXHR.fail( s.error ); + + // Get transport + transport = inspectPrefiltersOrTransports( transports, s, options, jqXHR ); + + // If no transport, we auto-abort + if ( !transport ) { + done( -1, "No Transport" ); + } else { + jqXHR.readyState = 1; + + // Send global event + if ( fireGlobals ) { + globalEventContext.trigger( "ajaxSend", [ jqXHR, s ] ); + } + + // If request was aborted inside ajaxSend, stop there + if ( completed ) { + return jqXHR; + } + + // Timeout + if ( s.async && s.timeout > 0 ) { + timeoutTimer = window.setTimeout( function() { + jqXHR.abort( "timeout" ); + }, s.timeout ); + } + + try { + completed = false; + transport.send( requestHeaders, done ); + } catch ( e ) { + + // Rethrow post-completion exceptions + if ( completed ) { + throw e; + } + + // Propagate others as results + done( -1, e ); + } + } + + // Callback for when everything is done + function done( status, nativeStatusText, responses, headers ) { + var isSuccess, success, error, response, modified, + statusText = nativeStatusText; + + // Ignore repeat invocations + if ( completed ) { + return; + } + + completed = true; + + // Clear timeout if it exists + if ( timeoutTimer ) { + window.clearTimeout( timeoutTimer ); + } + + // Dereference transport for early garbage collection + // (no matter how long the jqXHR object will be used) + transport = undefined; + + // Cache response headers + responseHeadersString = headers || ""; + + // Set readyState + jqXHR.readyState = status > 0 ? 4 : 0; + + // Determine if successful + isSuccess = status >= 200 && status < 300 || status === 304; + + // Get response data + if ( responses ) { + response = ajaxHandleResponses( s, jqXHR, responses ); + } + + // Use a noop converter for missing script but not if jsonp + if ( !isSuccess && + jQuery.inArray( "script", s.dataTypes ) > -1 && + jQuery.inArray( "json", s.dataTypes ) < 0 ) { + s.converters[ "text script" ] = function() {}; + } + + // Convert no matter what (that way responseXXX fields are always set) + response = ajaxConvert( s, response, jqXHR, isSuccess ); + + // If successful, handle type chaining + if ( isSuccess ) { + + // Set the If-Modified-Since and/or If-None-Match header, if in ifModified mode. + if ( s.ifModified ) { + modified = jqXHR.getResponseHeader( "Last-Modified" ); + if ( modified ) { + jQuery.lastModified[ cacheURL ] = modified; + } + modified = jqXHR.getResponseHeader( "etag" ); + if ( modified ) { + jQuery.etag[ cacheURL ] = modified; + } + } + + // if no content + if ( status === 204 || s.type === "HEAD" ) { + statusText = "nocontent"; + + // if not modified + } else if ( status === 304 ) { + statusText = "notmodified"; + + // If we have data, let's convert it + } else { + statusText = response.state; + success = response.data; + error = response.error; + isSuccess = !error; + } + } else { + + // Extract error from statusText and normalize for non-aborts + error = statusText; + if ( status || !statusText ) { + statusText = "error"; + if ( status < 0 ) { + status = 0; + } + } + } + + // Set data for the fake xhr object + jqXHR.status = status; + jqXHR.statusText = ( nativeStatusText || statusText ) + ""; + + // Success/Error + if ( isSuccess ) { + deferred.resolveWith( callbackContext, [ success, statusText, jqXHR ] ); + } else { + deferred.rejectWith( callbackContext, [ jqXHR, statusText, error ] ); + } + + // Status-dependent callbacks + jqXHR.statusCode( statusCode ); + statusCode = undefined; + + if ( fireGlobals ) { + globalEventContext.trigger( isSuccess ? "ajaxSuccess" : "ajaxError", + [ jqXHR, s, isSuccess ? success : error ] ); + } + + // Complete + completeDeferred.fireWith( callbackContext, [ jqXHR, statusText ] ); + + if ( fireGlobals ) { + globalEventContext.trigger( "ajaxComplete", [ jqXHR, s ] ); + + // Handle the global AJAX counter + if ( !( --jQuery.active ) ) { + jQuery.event.trigger( "ajaxStop" ); + } + } + } + + return jqXHR; + }, + + getJSON: function( url, data, callback ) { + return jQuery.get( url, data, callback, "json" ); + }, + + getScript: function( url, callback ) { + return jQuery.get( url, undefined, callback, "script" ); + } +} ); + +jQuery.each( [ "get", "post" ], function( _i, method ) { + jQuery[ method ] = function( url, data, callback, type ) { + + // Shift arguments if data argument was omitted + if ( isFunction( data ) ) { + type = type || callback; + callback = data; + data = undefined; + } + + // The url can be an options object (which then must have .url) + return jQuery.ajax( jQuery.extend( { + url: url, + type: method, + dataType: type, + data: data, + success: callback + }, jQuery.isPlainObject( url ) && url ) ); + }; +} ); + +jQuery.ajaxPrefilter( function( s ) { + var i; + for ( i in s.headers ) { + if ( i.toLowerCase() === "content-type" ) { + s.contentType = s.headers[ i ] || ""; + } + } +} ); + + +jQuery._evalUrl = function( url, options, doc ) { + return jQuery.ajax( { + url: url, + + // Make this explicit, since user can override this through ajaxSetup (#11264) + type: "GET", + dataType: "script", + cache: true, + async: false, + global: false, + + // Only evaluate the response if it is successful (gh-4126) + // dataFilter is not invoked for failure responses, so using it instead + // of the default converter is kludgy but it works. + converters: { + "text script": function() {} + }, + dataFilter: function( response ) { + jQuery.globalEval( response, options, doc ); + } + } ); +}; + + +jQuery.fn.extend( { + wrapAll: function( html ) { + var wrap; + + if ( this[ 0 ] ) { + if ( isFunction( html ) ) { + html = html.call( this[ 0 ] ); + } + + // The elements to wrap the target around + wrap = jQuery( html, this[ 0 ].ownerDocument ).eq( 0 ).clone( true ); + + if ( this[ 0 ].parentNode ) { + wrap.insertBefore( this[ 0 ] ); + } + + wrap.map( function() { + var elem = this; + + while ( elem.firstElementChild ) { + elem = elem.firstElementChild; + } + + return elem; + } ).append( this ); + } + + return this; + }, + + wrapInner: function( html ) { + if ( isFunction( html ) ) { + return this.each( function( i ) { + jQuery( this ).wrapInner( html.call( this, i ) ); + } ); + } + + return this.each( function() { + var self = jQuery( this ), + contents = self.contents(); + + if ( contents.length ) { + contents.wrapAll( html ); + + } else { + self.append( html ); + } + } ); + }, + + wrap: function( html ) { + var htmlIsFunction = isFunction( html ); + + return this.each( function( i ) { + jQuery( this ).wrapAll( htmlIsFunction ? html.call( this, i ) : html ); + } ); + }, + + unwrap: function( selector ) { + this.parent( selector ).not( "body" ).each( function() { + jQuery( this ).replaceWith( this.childNodes ); + } ); + return this; + } +} ); + + +jQuery.expr.pseudos.hidden = function( elem ) { + return !jQuery.expr.pseudos.visible( elem ); +}; +jQuery.expr.pseudos.visible = function( elem ) { + return !!( elem.offsetWidth || elem.offsetHeight || elem.getClientRects().length ); +}; + + + + +jQuery.ajaxSettings.xhr = function() { + try { + return new window.XMLHttpRequest(); + } catch ( e ) {} +}; + +var xhrSuccessStatus = { + + // File protocol always yields status code 0, assume 200 + 0: 200, + + // Support: IE <=9 only + // #1450: sometimes IE returns 1223 when it should be 204 + 1223: 204 + }, + xhrSupported = jQuery.ajaxSettings.xhr(); + +support.cors = !!xhrSupported && ( "withCredentials" in xhrSupported ); +support.ajax = xhrSupported = !!xhrSupported; + +jQuery.ajaxTransport( function( options ) { + var callback, errorCallback; + + // Cross domain only allowed if supported through XMLHttpRequest + if ( support.cors || xhrSupported && !options.crossDomain ) { + return { + send: function( headers, complete ) { + var i, + xhr = options.xhr(); + + xhr.open( + options.type, + options.url, + options.async, + options.username, + options.password + ); + + // Apply custom fields if provided + if ( options.xhrFields ) { + for ( i in options.xhrFields ) { + xhr[ i ] = options.xhrFields[ i ]; + } + } + + // Override mime type if needed + if ( options.mimeType && xhr.overrideMimeType ) { + xhr.overrideMimeType( options.mimeType ); + } + + // X-Requested-With header + // For cross-domain requests, seeing as conditions for a preflight are + // akin to a jigsaw puzzle, we simply never set it to be sure. + // (it can always be set on a per-request basis or even using ajaxSetup) + // For same-domain requests, won't change header if already provided. + if ( !options.crossDomain && !headers[ "X-Requested-With" ] ) { + headers[ "X-Requested-With" ] = "XMLHttpRequest"; + } + + // Set headers + for ( i in headers ) { + xhr.setRequestHeader( i, headers[ i ] ); + } + + // Callback + callback = function( type ) { + return function() { + if ( callback ) { + callback = errorCallback = xhr.onload = + xhr.onerror = xhr.onabort = xhr.ontimeout = + xhr.onreadystatechange = null; + + if ( type === "abort" ) { + xhr.abort(); + } else if ( type === "error" ) { + + // Support: IE <=9 only + // On a manual native abort, IE9 throws + // errors on any property access that is not readyState + if ( typeof xhr.status !== "number" ) { + complete( 0, "error" ); + } else { + complete( + + // File: protocol always yields status 0; see #8605, #14207 + xhr.status, + xhr.statusText + ); + } + } else { + complete( + xhrSuccessStatus[ xhr.status ] || xhr.status, + xhr.statusText, + + // Support: IE <=9 only + // IE9 has no XHR2 but throws on binary (trac-11426) + // For XHR2 non-text, let the caller handle it (gh-2498) + ( xhr.responseType || "text" ) !== "text" || + typeof xhr.responseText !== "string" ? + { binary: xhr.response } : + { text: xhr.responseText }, + xhr.getAllResponseHeaders() + ); + } + } + }; + }; + + // Listen to events + xhr.onload = callback(); + errorCallback = xhr.onerror = xhr.ontimeout = callback( "error" ); + + // Support: IE 9 only + // Use onreadystatechange to replace onabort + // to handle uncaught aborts + if ( xhr.onabort !== undefined ) { + xhr.onabort = errorCallback; + } else { + xhr.onreadystatechange = function() { + + // Check readyState before timeout as it changes + if ( xhr.readyState === 4 ) { + + // Allow onerror to be called first, + // but that will not handle a native abort + // Also, save errorCallback to a variable + // as xhr.onerror cannot be accessed + window.setTimeout( function() { + if ( callback ) { + errorCallback(); + } + } ); + } + }; + } + + // Create the abort callback + callback = callback( "abort" ); + + try { + + // Do send the request (this may raise an exception) + xhr.send( options.hasContent && options.data || null ); + } catch ( e ) { + + // #14683: Only rethrow if this hasn't been notified as an error yet + if ( callback ) { + throw e; + } + } + }, + + abort: function() { + if ( callback ) { + callback(); + } + } + }; + } +} ); + + + + +// Prevent auto-execution of scripts when no explicit dataType was provided (See gh-2432) +jQuery.ajaxPrefilter( function( s ) { + if ( s.crossDomain ) { + s.contents.script = false; + } +} ); + +// Install script dataType +jQuery.ajaxSetup( { + accepts: { + script: "text/javascript, application/javascript, " + + "application/ecmascript, application/x-ecmascript" + }, + contents: { + script: /\b(?:java|ecma)script\b/ + }, + converters: { + "text script": function( text ) { + jQuery.globalEval( text ); + return text; + } + } +} ); + +// Handle cache's special case and crossDomain +jQuery.ajaxPrefilter( "script", function( s ) { + if ( s.cache === undefined ) { + s.cache = false; + } + if ( s.crossDomain ) { + s.type = "GET"; + } +} ); + +// Bind script tag hack transport +jQuery.ajaxTransport( "script", function( s ) { + + // This transport only deals with cross domain or forced-by-attrs requests + if ( s.crossDomain || s.scriptAttrs ) { + var script, callback; + return { + send: function( _, complete ) { + script = jQuery( " + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Operations

+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/dev/extensions.html b/docs/build/html/dev/extensions.html new file mode 100644 index 000000000..30bc58701 --- /dev/null +++ b/docs/build/html/dev/extensions.html @@ -0,0 +1,1036 @@ + + + + + + + Developer Documentation — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Developer Documentation

+

MLX provides a open and flexible backend to which users may add operations +and specialized implementations without much hassle. While the library supplies +efficient operations that can be used and composed for any number of +applications, there may arise cases where new functionalities or highly +optimized implementations are needed. For such cases, you may design and +implement your own operations that link to and build on top of mlx.core. +We will introduce the inner-workings of MLX and go over a simple example to +learn the steps involved in adding new operations to MLX with your own CPU +and GPU implementations.

+
+

Introducing the Example

+

Let’s say that you would like an operation that takes in two arrays, +x and y, scales them both by some coefficents alpha and beta +respectively, and then adds them together to get the result +z = alpha * x + beta * y. Well, you can very easily do that by just +writing out a function as follows:

+
import mlx.core as mx
+
+def simple_axpby(x: mx.array, y: mx.array, alpha: float, beta: float) -> mx.array:
+    return alpha * x + beta * y
+
+
+

This function performs that operation while leaving the implementations and +differentiation to MLX.

+

However, you work with vector math libraries often and realize that the +axpby routine defines the same operation Y = (alpha * X) + (beta * Y). +You would really like the part of your applications that does this operation +on the CPU to be very fast - so you decide that you want it to rely on the +axpby routine provided by the Accelerate framework. Continuing to impose +our assumptions on to you, let’s also assume that you want to learn how add +your own implementation for the gradients of your new operation while going +over the ins-and-outs of the MLX framework.

+

Well, what a coincidence! You are in the right place. Over the course of this +example, we will learn:

+
    +

  • The structure of the MLX library from the frontend API to the backend implementations.

    • +

  • How to implement your own CPU backend that redirects to Accelerate when appropriate (and a fallback if needed).

    • +

  • How to implement your own GPU implementation using metal.

    • +

  • How to add your own vjp and jvp.

    • +

  • How to build your implementations, link them to MLX, and bind them to python.

    • +
+
+
+

Operations and Primitives

+

In one sentence, operations in MLX build the computation graph, and primitives +provide the rules for evaluation and transformations of said graph. Let’s start +by discussing operations in more detail.

+
+

Operations

+

Operations are the frontend functions that operate on arrays. They are defined +in the C++ API (Operations) and then we provide bindings to these +operations in the Python API (Operations).

+

We would like an operation, axpby() that takes in two arrays x and y, +and two scalars, alpha and beta. This is how we would define it in the +C++ API:

+
/**
+*  Scale and sum two vectors elementwise
+*  z = alpha * x + beta * y
+*
+*  Follow numpy style broadcasting between x and y
+*  Inputs are upcasted to floats if needed
+**/
+array axpby(
+    const array& x, // Input array x
+    const array& y, // Input array y
+    const float alpha, // Scaling factor for x
+    const float beta, // Scaling factor for y
+    StreamOrDevice s = {} // Stream on which to schedule the operation
+);
+
+
+

This operation itself can call other operations within it if needed. So, the +simplest way to go about implementing this operation would be do so in terms +of existing operations.

+
array axpby(
+    const array& x, // Input array x
+    const array& y, // Input array y
+    const float alpha, // Scaling factor for x
+    const float beta, // Scaling factor for y
+    StreamOrDevice s /* = {} */ // Stream on which to schedule the operation
+) {
+    // Scale x and y on the provided stream
+    auto ax = multiply(array(alpha), x, s);
+    auto by = multiply(array(beta), y, s);
+
+    // Add and return
+    return add(ax, by, s);
+}
+
+
+

However, as we discussed earlier, this is not our goal. The operations themselves +do not contain the implementations that act on the data, nor do they contain the +rules of transformations. Rather, they are an easy to use interface that build +on top of the building blocks we call Primitive.

+
+
+

Primitives

+

A Primitive is part of the computation graph of an array. It +defines how to create an output given a set of input array . Further, +a Primitive is a class that contains rules on how it is evaluated +on the CPU or GPU, and how it acts under transformations such as vjp and +jvp. These words on their own can be a bit abstract, so lets take a step +back and go to our example to give ourselves a more concrete image.

+
class Axpby : public Primitive {
+  public:
+    explicit Axpby(Stream stream, float alpha, float beta)
+        : Primitive(stream), alpha_(alpha), beta_(beta){};
+
+    /**
+    * A primitive must know how to evaluate itself on the CPU/GPU
+    * for the given inputs and populate the output array.
+    *
+    * To avoid unecessary allocations, the evaluation function
+    * is responsible for allocating space for the array.
+    */
+    void eval_cpu(const std::vector<array>& inputs, array& out) override;
+    void eval_gpu(const std::vector<array>& inputs, array& out) override;
+
+    /** The Jacobian-vector product. */
+    array jvp(
+        const std::vector<array>& primals,
+        const std::vector<array>& tangents,
+        const std::vector<int>& argnums) override;
+
+    /** The vector-Jacobian product. */
+    std::vector<array> vjp(
+        const std::vector<array>& primals,
+        const array& cotan,
+        const std::vector<int>& argnums) override;
+
+    /**
+    * The primitive must know how to vectorize itself accross
+    * the given axes. The output is a pair containing the array
+    * representing the vectorized computation and the axis which
+    * corresponds to the output vectorized dimension.
+    */
+    std::pair<array, int> vmap(
+        const std::vector<array>& inputs,
+        const std::vector<int>& axes) override;
+
+    /** Print the primitive. */
+    void print(std::ostream& os) override {
+        os << "Axpby";
+    }
+
+    /** Equivalence check **/
+    bool is_equivalent(const Primitive& other) const override;
+
+  private:
+    float alpha_;
+    float beta_;
+
+    /** Fall back implementation for evaluation on CPU */
+    void eval(const std::vector<array>& inputs, array& out);
+};
+
+
+

The Axpby class derives from the base Primitive class and +follows the above demonstrated interface. Axpby treats alpha and +beta as parameters. It then provides implementations of how the array out +is produced given inputs through Axpby::eval_cpu() and +Axpby::eval_gpu(). Further, it provides rules of transformations in +Axpby::jvp(), Axpby::vjp(), and Axpby::vmap().

+
+
+

Using the Primitives

+

Operations can use this Primitive to add a new array to +the computation graph. An array can be constructed by providing its +data type, shape, the Primitive that computes it, and the +array inputs that are passed to the primitive.

+

Let’s re-implement our operation now in terms of our Axpby primitive.

+
array axpby(
+    const array& x, // Input array x
+    const array& y, // Input array y
+    const float alpha, // Scaling factor for x
+    const float beta, // Scaling factor for y
+    StreamOrDevice s /* = {} */ // Stream on which to schedule the operation
+) {
+    // Promote dtypes between x and y as needed
+    auto promoted_dtype = promote_types(x.dtype(), y.dtype());
+
+    // Upcast to float32 for non-floating point inputs x and y
+    auto out_dtype = is_floating_point(promoted_dtype)
+        ? promoted_dtype
+        : promote_types(promoted_dtype, float32);
+
+    // Cast x and y up to the determined dtype (on the same stream s)
+    auto x_casted = astype(x, out_dtype, s);
+    auto y_casted = astype(y, out_dtype, s);
+
+    // Broadcast the shapes of x and y (on the same stream s)
+    auto broadcasted_inputs = broadcast_arrays({x_casted, y_casted}, s);
+    auto out_shape = broadcasted_inputs[0].shape();
+
+    // Construct the array as the output of the Axpby primitive
+    // with the broadcasted and upcasted arrays as inputs
+    return array(
+        /* const std::vector<int>& shape = */ out_shape,
+        /* Dtype dtype = */ out_dtype,
+        /* std::unique_ptr<Primitive> primitive = */
+        std::make_unique<Axpby>(to_stream(s), alpha, beta),
+        /* const std::vector<array>& inputs = */ broadcasted_inputs);
+}
+
+
+

This operation now handles the following:

+
    +

  1. Upcast inputs and resolve the the output data type.

    2. +

  2. Broadcast the inputs and resolve the output shape.

    3. +

  3. Construct the primitive Axpby using the given stream, alpha, and beta.

    4. +

  4. Construct the output array using the primitive and the inputs.

    5. +
+
+
+
+

Implementing the Primitive

+

No computation happens when we call the operation alone. In effect, the +operation only builds the computation graph. When we evaluate the output +array, MLX schedules the execution of the computation graph, and calls +Axpby::eval_cpu() or Axpby::eval_gpu() depending on the +stream/device specified by the user.

+
+

Warning

+

When Primitive::eval_cpu() or Primitive::eval_gpu() are called, +no memory has been allocated for the output array. It falls on the implementation +of these functions to allocate memory as needed

+
+
+

Implementing the CPU Backend

+

Let’s start by trying to implement a naive and generic version of +Axpby::eval_cpu(). We declared this as a private member function of +Axpby earlier called Axpby::eval().

+

Our naive method will go over each element of the output array, find the +corresponding input elements of x and y and perform the operation +pointwise. This is captured in the templated function axpby_impl().

+
template <typename T>
+void axpby_impl(
+        const array& x,
+        const array& y,
+        array& out,
+        float alpha_,
+        float beta_) {
+    // We only allocate memory when we are ready to fill the output
+    // malloc_or_wait synchronously allocates available memory
+    // There may be a wait executed here if the allocation is requested
+    // under memory-pressured conditions
+    out.set_data(allocator::malloc_or_wait(out.nbytes()));
+
+    // Collect input and output data pointers
+    const T* x_ptr = x.data<T>();
+    const T* y_ptr = y.data<T>();
+    T* out_ptr = out.data<T>();
+
+    // Cast alpha and beta to the relevant types
+    T alpha = static_cast<T>(alpha_);
+    T beta = static_cast<T>(beta_);
+
+    // Do the elementwise operation for each output
+    for (size_t out_idx = 0; out_idx < out.size(); out_idx++) {
+        // Map linear indices to offsets in x and y
+        auto x_offset = elem_to_loc(out_idx, x.shape(), x.strides());
+        auto y_offset = elem_to_loc(out_idx, y.shape(), y.strides());
+
+        // We allocate the output to be contiguous and regularly strided
+        // (defaults to row major) and hence it doesn't need additonal mapping
+        out_ptr[out_idx] = alpha * x_ptr[x_offset] + beta * y_ptr[y_offset];
+    }
+}
+
+
+

Now, we would like our implementation to be able to do this pointwise operation +for all incoming floating point arrays. Accordingly, we add dispatches for +float32, float16, bfloat16 and complex64. We throw an error +if we encounter an unexpected type.

+
/** Fall back implementation for evaluation on CPU */
+void Axpby::eval(const std::vector<array>& inputs, array& out) {
+    // Check the inputs (registered in the op while contructing the out array)
+    assert(inputs.size() == 2);
+    auto& x = inputs[0];
+    auto& y = inputs[1];
+
+    // Dispatch to the correct dtype
+    if (out.dtype() == float32) {
+        return axpby_impl<float>(x, y, out, alpha_, beta_);
+    } else if (out.dtype() == float16) {
+        return axpby_impl<float16_t>(x, y, out, alpha_, beta_);
+    } else if (out.dtype() == bfloat16) {
+        return axpby_impl<bfloat16_t>(x, y, out, alpha_, beta_);
+    } else if (out.dtype() == complex64) {
+        return axpby_impl<complex64_t>(x, y, out, alpha_, beta_);
+    } else {
+        throw std::runtime_error(
+            "Axpby is only supported for floating point types.");
+    }
+}
+
+
+

We have a fallback implementation! Now, to do what we are really here to do. +Remember we wanted to use the axpby routine provided by the Accelerate +framework? Well, there are 3 complications to keep in mind:

+
    +

  1. Accelerate does not provide implementations of axpby for half precision +floats. We can only direct to it for float32 types

    2. +

  2. Accelerate assumes the inputs x and y are contiguous and all elements +have fixed strides between them. Possibly due to broadcasts and transposes, +we aren’t guaranteed that the inputs fit this requirement. We can +only direct to Accelerate if both x and y are row contiguous or +column contiguous.

    3. +

  3. Accelerate performs the routine Y = (alpha * X) + (beta * Y) inplace. +MLX expects to write out the answer to a new array. We must copy the elements +of y into the output array and use that as an input to axpby

    4. +
+

Let’s write out an implementation that uses Accelerate in the right conditions. +It must simply allocate data for the output, copy elements of y into it, +and then call the catlas_saxpby() from accelerate.

+
template <typename T>
+void axpby_impl_accelerate(
+        const array& x,
+        const array& y,
+        array& out,
+        float alpha_,
+        float beta_) {
+    // Accelerate library provides catlas_saxpby which does
+    // Y = (alpha * X) + (beta * Y) in place
+    // To use it, we first copy the data in y over to the output array
+
+    // This specialization requires both x and y be contiguous in the same mode
+    // i.e: corresponding linear indices in both point to corresponding elements
+    // The data in the output array is allocated to match the strides in y
+    // such that x, y, and out are contiguous in the same mode and
+    // no transposition is needed
+    out.set_data(
+        allocator::malloc_or_wait(y.data_size() * out.itemsize()),
+        y.data_size(),
+        y.strides(),
+        y.flags());
+
+    // We then copy over the elements using the contiguous vector specialization
+    copy_inplace(y, out, CopyType::Vector);
+
+    // Get x and y pointers for catlas_saxpby
+    const T* x_ptr = x.data<T>();
+    T* y_ptr = out.data<T>();
+
+    T alpha = static_cast<T>(alpha_);
+    T beta = static_cast<T>(beta_);
+
+    // Call the inplace accelerate operator
+    catlas_saxpby(
+        /* N = */ out.size(),
+        /* ALPHA = */ alpha,
+        /* X = */ x_ptr,
+        /* INCX = */ 1,
+        /* BETA = */ beta,
+        /* Y = */ y_ptr,
+        /* INCY = */ 1);
+}
+
+
+

Great! But what about the inputs that do not fit the criteria for accelerate? +Luckily, we can always just direct back to Axpby::eval().

+

With this in mind, lets finally implement our Axpby::eval_cpu().

+
/** Evaluate primitive on CPU using accelerate specializations */
+void Axpby::eval_cpu(const std::vector<array>& inputs, array& out) {
+    assert(inputs.size() == 2);
+    auto& x = inputs[0];
+    auto& y = inputs[1];
+
+    // Accelerate specialization for contiguous single precision float arrays
+    if (out.dtype() == float32 &&
+        ((x.flags().row_contiguous && y.flags().row_contiguous) ||
+        (x.flags().col_contiguous && y.flags().col_contiguous))) {
+        axpby_impl_accelerate<float>(x, y, out, alpha_, beta_);
+        return;
+    }
+
+    // Fall back to common backend if specializations are not available
+    eval(inputs, out);
+}
+
+
+

We have now hit a milestone! Just this much is enough to run the operation +axpby() on a CPU stream!

+

If you do not plan on running the operation on the GPU or using transforms on +computation graphs that contain Axpby, you can stop implementing the +primitive here and enjoy the speed-ups you get from the Accelerate library.

+
+
+

Implementing the GPU Backend

+

Apple silicon devices address their GPUs using the Metal shading language, and +all GPU kernels in MLX are written using metal.

+
+

Note

+

Here are some helpful resources if you are new to metal!

+ +
+

Let’s keep the GPU algorithm simple. We will launch exactly as many threads +as there are elements in the output. Each thread will pick the element it needs +from x and y, do the pointwise operation, and then update its assigned +element in the output.

+
template <typename T>
+[[kernel]] void axpby_general(
+        device const T* x [[buffer(0)]],
+        device const T* y [[buffer(1)]],
+        device T* out [[buffer(2)]],
+        constant const float& alpha [[buffer(3)]],
+        constant const float& beta [[buffer(4)]],
+        constant const int* shape [[buffer(5)]],
+        constant const size_t* x_strides [[buffer(6)]],
+        constant const size_t* y_strides [[buffer(7)]],
+        constant const int& ndim [[buffer(8)]],
+        uint index [[thread_position_in_grid]]) {
+    // Convert linear indices to offsets in array
+    auto x_offset = elem_to_loc(index, shape, x_strides, ndim);
+    auto y_offset = elem_to_loc(index, shape, y_strides, ndim);
+
+    // Do the operation and update the output
+    out[index] =
+        static_cast<T>(alpha) * x[x_offset] + static_cast<T>(beta) * y[y_offset];
+}
+
+
+

We then need to instantiate this template for all floating point types and give +each instantiation a unique host name so we can identify the right kernel for +each data type.

+
#define instantiate_axpby(type_name, type)              \
+    template [[host_name("axpby_general_" #type_name)]] \
+    [[kernel]] void axpby_general<type>(                \
+        device const type* x [[buffer(0)]],             \
+        device const type* y [[buffer(1)]],             \
+        device type* out [[buffer(2)]],                 \
+        constant const float& alpha [[buffer(3)]],      \
+        constant const float& beta [[buffer(4)]],       \
+        constant const int* shape [[buffer(5)]],        \
+        constant const size_t* x_strides [[buffer(6)]], \
+        constant const size_t* y_strides [[buffer(7)]], \
+        constant const int& ndim [[buffer(8)]],         \
+        uint index [[thread_position_in_grid]]);
+
+instantiate_axpby(float32, float);
+instantiate_axpby(float16, half);
+instantiate_axpby(bflot16, bfloat16_t);
+instantiate_axpby(complex64, complex64_t);
+
+
+

This kernel will be compiled into a metal library mlx_ext.metallib as we +will see later in Building with CMake. In the following example, we +assume that the library mlx_ext.metallib will always be co-located with +the executable/ shared-library calling the register_library() function. +The register_library() function takes the library’s name and potential +path (or in this case, a function that can produce the path of the metal +library) and tries to load that library if it hasn’t already been registered +by the relevant static mlx::core::metal::Device object. This is why, +it is important to package your C++ library with the metal library. We will +go over this process in more detail later.

+

The logic to determine the kernel, set the inputs, resolve the grid dimensions +and dispatch it to the GPU are contained in Axpby::eval_gpu() as shown +below.

+
/** Evaluate primitive on GPU */
+void Axpby::eval_gpu(const std::vector<array>& inputs, array& out) {
+    // Prepare inputs
+    assert(inputs.size() == 2);
+    auto& x = inputs[0];
+    auto& y = inputs[1];
+
+    // Each primitive carries the stream it should execute on
+    // and each stream carries its device identifiers
+    auto& s = stream();
+    // We get the needed metal device using the stream
+    auto& d = metal::device(s.device);
+
+    // Allocate output memory
+    out.set_data(allocator::malloc_or_wait(out.nbytes()));
+
+    // Resolve name of kernel (corresponds to axpby.metal)
+    std::ostringstream kname;
+    kname << "axpby_" << "general_" << type_to_name(out);
+
+    // Make sure the metal library is available and look for it
+    // in the same folder as this executable if needed
+    d.register_library("mlx_ext", metal::get_colocated_mtllib_path);
+
+    // Make a kernel from this metal library
+    auto kernel = d.get_kernel(kname.str(), "mlx_ext");
+
+    // Prepare to encode kernel
+    auto compute_encoder = d.get_command_encoder(s.index);
+    compute_encoder->setComputePipelineState(kernel);
+
+    // Kernel parameters are registered with buffer indices corresponding to
+    // those in the kernel decelaration at axpby.metal
+    int ndim = out.ndim();
+    size_t nelem = out.size();
+
+    // Encode input arrays to kernel
+    set_array_buffer(compute_encoder, x, 0);
+    set_array_buffer(compute_encoder, y, 1);
+
+    // Encode output arrays to kernel
+    set_array_buffer(compute_encoder, out, 2);
+
+    // Encode alpha and beta
+    compute_encoder->setBytes(&alpha_, sizeof(float), 3);
+    compute_encoder->setBytes(&beta_, sizeof(float), 4);
+
+    // Encode shape, strides and ndim
+    compute_encoder->setBytes(x.shape().data(), ndim * sizeof(int), 5);
+    compute_encoder->setBytes(x.strides().data(), ndim * sizeof(size_t), 6);
+    compute_encoder->setBytes(y.strides().data(), ndim * sizeof(size_t), 7);
+    compute_encoder->setBytes(&ndim, sizeof(int), 8);
+
+    // We launch 1 thread for each input and make sure that the number of
+    // threads in any given threadgroup is not higher than the max allowed
+    size_t tgp_size = std::min(nelem, kernel->maxTotalThreadsPerThreadgroup());
+
+    // Fix the 3D size of each threadgroup (in terms of threads)
+    MTL::Size group_dims = MTL::Size(tgp_size, 1, 1);
+
+    // Fix the 3D size of the launch grid (in terms of threads)
+    MTL::Size grid_dims = MTL::Size(nelem, 1, 1);
+
+    // Launch the grid with the given number of threads divded among
+    // the given threadgroups
+    compute_encoder->dispatchThreads(grid_dims, group_dims);
+}
+
+
+

We can now call the axpby() operation on both the CPU and the GPU!

+

A few things to note about MLX and metal before moving on. MLX keeps track +of the active compute_encoder. We rely on d.get_command_encoder() +to give us the active metal compute command encoder instead of building a +new one and calling compute_encoder->end_encoding() at the end. +MLX keeps adding kernels (compute pipelines) to the active command encoder +until some specified limit is hit or the compute encoder needs to be flushed +for synchronization. MLX also handles enqueuing and commiting the associated +command buffers as needed. We suggest taking a deeper dive into +metal::Device if you would like to study this routine further.

+
+
+

Primitive Transforms

+

Now that we have come this far, let’s also learn how to add implementations to +transformations in a Primitive. These transformations can be built on +top of our operations, including the one we just defined now. Which then gives +us the following Axpby::jvp() and Axpby::vjp() implementations.

+
/** The Jacobian-vector product. */
+array Axpby::jvp(
+        const std::vector<array>& primals,
+        const std::vector<array>& tangents,
+        const std::vector<int>& argnums) {
+    // Forward mode diff that pushes along the tangents
+    // The jvp transform on the the primitive can built with ops
+    // that are scheduled on the same stream as the primtive
+
+    // If argnums = {0}, we only push along x in which case the
+    // jvp is just the tangent scaled by alpha
+    // Similarly, if argnums = {1}, the jvp is just the tangent
+    // scaled by beta
+    if (argnums.size() > 1) {
+        auto scale = argnums[0] == 0 ? alpha_ : beta_;
+        auto scale_arr = array(scale, tangents[0].dtype());
+        return multiply(scale_arr, tangents[0], stream());
+    }
+    // If, argnums = {0, 1}, we take contributions from both
+    // which gives us jvp = tangent_x * alpha + tangent_y * beta
+    else {
+        return axpby(tangents[0], tangents[1], alpha_, beta_, stream());
+    }
+}
+
+
+
/** The vector-Jacobian product. */
+std::vector<array> Axpby::vjp(
+        const std::vector<array>& primals,
+        const array& cotan,
+        const std::vector<int>& argnums) {
+    // Reverse mode diff
+    std::vector<array> vjps;
+    for (auto arg : argnums) {
+        auto scale = arg == 0 ? alpha_ : beta_;
+        auto scale_arr = array(scale, cotan.dtype());
+        vjps.push_back(multiply(scale_arr, cotan, stream()));
+    }
+    return vjps;
+}
+
+
+

Finally, you need not have a transformation fully defined to start using your +own Primitive.

+
/** Vectorize primitve along given axis */
+std::pair<array, int> Axpby::vmap(
+        const std::vector<array>& inputs,
+        const std::vector<int>& axes) {
+    throw std::runtime_error("Axpby has no vmap implementation.");
+}
+
+
+
+
+
+

Building and Binding

+

Let’s look at the overall directory structure first.

+
+
extensions
+
├── axpby
+
│ ├── axpby.cpp
+
│ ├── axpby.h
+
│ └── axpby.metal
+
├── mlx_sample_extensions
+
│ └── __init__.py
+
├── bindings.cpp
+
├── CMakeLists.txt
+
└── setup.py
+
+
    +

  • extensions/axpby/ defines the C++ extension library

    • +

  • extensions/mlx_sample_extensions sets out the strucutre for the +associated python package

    • +

  • extensions/bindings.cpp provides python bindings for our operation

    • +

  • extensions/CMakeLists.txt holds CMake rules to build the library and +python bindings

    • +

  • extensions/setup.py holds the setuptools rules to build and install +the python package

    • +
+
+

Binding to Python

+

We use PyBind11 to build a Python API for the C++ library. Since bindings +for all needed components such as mlx.core.array, mlx.core.stream, etc. +are already provided, adding our axpby() becomes very simple!

+
PYBIND11_MODULE(mlx_sample_extensions, m) {
+    m.doc() = "Sample C++ and metal extensions for MLX";
+
+    m.def(
+        "axpby",
+        &axpby,
+        "x"_a,
+        "y"_a,
+        py::pos_only(),
+        "alpha"_a,
+        "beta"_a,
+        py::kw_only(),
+        "stream"_a = py::none(),
+        R"pbdoc(
+            Scale and sum two vectors elementwise
+            ``z = alpha * x + beta * y``
+
+            Follows numpy style broadcasting between ``x`` and ``y``
+            Inputs are upcasted to floats if needed
+
+            Args:
+                x (array): Input array.
+                y (array): Input array.
+                alpha (float): Scaling factor for ``x``.
+                beta (float): Scaling factor for ``y``.
+
+            Returns:
+                array: ``alpha * x + beta * y``
+        )pbdoc");
+}
+
+
+

Most of the complexity in the above example comes from additional bells and +whistles such as the literal names and doc-strings.

+
+

Warning

+

mlx.core needs to be imported before importing +mlx_sample_extensions as defined by the pybind11 module above to +ensure that the casters for mlx.core components like +mlx.core.array are available.

+
+
+
+

Building with CMake

+

Building the C++ extension library itself is simple, it only requires that you +find_package(MLX CONFIG) and then link it to your library.

+
# Add library
+add_library(mlx_ext)
+
+# Add sources
+target_sources(
+    mlx_ext
+    PUBLIC
+    ${CMAKE_CURRENT_LIST_DIR}/axpby/axpby.cpp
+)
+
+# Add include headers
+target_include_directories(
+    mlx_ext PUBLIC ${CMAKE_CURRENT_LIST_DIR}
+)
+
+# Link to mlx
+target_link_libraries(mlx_ext PUBLIC mlx)
+
+
+

We also need to build the attached metal library. For convenience, we provide a +mlx_build_metallib() function that builds a .metallib target given +sources, headers, destinations, etc. (defined in cmake/extension.cmake and +automatically imported with MLX package).

+

Here is what that looks like in practice!

+
# Build metallib
+if(MLX_BUILD_METAL)
+
+mlx_build_metallib(
+    TARGET mlx_ext_metallib
+    TITLE mlx_ext
+    SOURCES ${CMAKE_CURRENT_LIST_DIR}/axpby/axpby.metal
+    INCLUDE_DIRS ${PROJECT_SOURCE_DIR} ${MLX_INCLUDE_DIRS}
+    OUTPUT_DIRECTORY ${CMAKE_LIBRARY_OUTPUT_DIRECTORY}
+)
+
+add_dependencies(
+    mlx_ext
+    mlx_ext_metallib
+)
+
+endif()
+
+
+

Finally, we build the Pybind11 bindings

+
pybind11_add_module(
+    mlx_sample_extensions
+    ${CMAKE_CURRENT_LIST_DIR}/bindings.cpp
+)
+target_link_libraries(mlx_sample_extensions PRIVATE mlx_ext)
+
+if(BUILD_SHARED_LIBS)
+    target_link_options(mlx_sample_extensions PRIVATE -Wl,-rpath,@loader_path)
+endif()
+
+
+
+
+

Building with setuptools

+

Once we have set out the CMake build rules as described above, we can use the +build utilities defined in mlx.extension for a simple build process.

+
from mlx import extension
+from setuptools import setup
+
+if __name__ == "__main__":
+    setup(
+        name="mlx_sample_extensions",
+        version="0.0.0",
+        description="Sample C++ and Metal extensions for MLX primitives.",
+        ext_modules=[extension.CMakeExtension("mlx_sample_extensions")],
+        cmdclass={"build_ext": extension.CMakeBuild},
+        packages = ["mlx_sample_extensions"],
+        package_dir = {"": "mlx_sample_extensions"},
+        package_data = {"mlx_sample_extensions" : ["*.so", "*.dylib", "*.metallib"]},
+        zip_safe=False,
+        python_requires=">=3.7",
+    )
+
+
+
+

Note

+

We treat extensions/mlx_sample_extensions as the package directory +even though it only contains a __init__.py to ensure the following:

+
    +

  • mlx.core is always imported before importing mlx_sample_extensions

    • +

  • The C++ extension library and the metal library are co-located with the python +bindings and copied together if the package is installed

    • +
+
+

You can build inplace for development using +python setup.py build_ext -j8 --inplace (in extensions/)

+

This will result in a directory structure as follows:

+
+
extensions
+
├── mlx_sample_extensions
+
│ ├── __init__.py
+
│ ├── libmlx_ext.dylib # C++ extension library
+
│ ├── mlx_ext.metallib # Metal library
+
│ └── mlx_sample_extensions.cpython-3x-darwin.so # Python Binding
+
…
+
+

When you try to install using the command python -m pip install . +(in extensions/), the package will be installed with the same strucutre as +extensions/mlx_sample_extensions and the C++ and metal library will be +copied along with the python binding since they are specified as package_data.

+
+
+
+

Usage

+

After installing the extension as described above, you should be able to simply +import the python package and play with it as you would any other MLX operation!

+

Let’s looks at a simple script and it’s results!

+
import mlx.core as mx
+from mlx_sample_extensions import axpby
+
+a = mx.ones((3, 4))
+b = mx.ones((3, 4))
+c = axpby(a, b, 4.0, 2.0, stream=mx.cpu)
+
+print(f"c shape: {c.shape}")
+print(f"c dtype: {c.dtype}")
+print(f"c correctness: {mx.all(c == 6.0).item()}")
+
+
+

Output:

+
c shape: [3, 4]
+c dtype: float32
+c correctness: True
+
+
+
+

Results

+

Let’s run a quick benchmark and see how our new axpby operation compares +with the naive simple_axpby() we defined at first on the CPU.

+
import mlx.core as mx
+from mlx_sample_extensions import axpby
+import time
+
+mx.set_default_device(mx.cpu)
+
+def simple_axpby(x: mx.array, y: mx.array, alpha: float, beta: float) -> mx.array:
+    return alpha * x + beta * y
+
+M = 256
+N = 512
+
+x = mx.random.normal((M, N))
+y = mx.random.normal((M, N))
+alpha = 4.0
+beta = 2.0
+
+mx.eval((x, y))
+
+def bench(f):
+    # Warm up
+    for i in range(100):
+        z = f(x, y, alpha, beta)
+        mx.eval(z)
+
+    # Timed run
+    s = time.time()
+    for i in range(5000):
+        z = f(x, y, alpha, beta)
+        mx.eval(z)
+    e = time.time()
+    return e - s
+
+simple_time = bench(simple_axpby)
+custom_time = bench(axpby)
+
+print(f"Simple axpby: {simple_time:.3f} s | Custom axpby: {custom_time:.3f} s")
+
+
+

Results:

+
Simple axpby: 0.114 s | Custom axpby: 0.109 s
+
+
+

We see some modest improvements right away!

+

This operation is now good to be used to build other operations, +in mlx.nn.Module calls, and also as a part of graph +transformations such as grad() and simplify()!

+
+
+
+

Scripts

+
+

Download the code

+

The full example code is available in mlx-examples.

+
+
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/examples/linear_regression.html b/docs/build/html/examples/linear_regression.html new file mode 100644 index 000000000..ff219032e --- /dev/null +++ b/docs/build/html/examples/linear_regression.html @@ -0,0 +1,202 @@ + + + + + + + Linear Regression — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Linear Regression

+

Let’s implement a basic linear regression model as a starting point to +learn MLX. First import the core package and setup some problem metadata:

+
import mlx.core as mx
+
+num_features = 100
+num_examples = 1_000
+num_iters = 10_000  # iterations of SGD
+lr = 0.01  # learning rate for SGD
+
+
+

We’ll generate a synthetic dataset by:

+
    +

  1. Sampling the design matrix X.

    2. +

  2. Sampling a ground truth parameter vector w_star.

    3. +

  3. Compute the dependent values y by adding Gaussian noise to X @ w_star.

    4. +
+
# True parameters
+w_star = mx.random.normal((num_features,))
+
+# Input examples (design matrix)
+X = mx.random.normal((num_examples, num_features))
+
+# Noisy labels
+eps = 1e-2 * mx.random.normal((num_examples,))
+y = X @ w_star + eps
+
+
+

We will use SGD to find the optimal weights. To start, define the squared loss +and get the gradient function of the loss with respect to the parameters.

+
def loss_fn(w):
+    return 0.5 * mx.mean(mx.square(X @ w - y))
+
+grad_fn = mx.grad(loss_fn)
+
+
+

Start the optimization by initializing the parameters w randomly. Then +repeatedly update the parameters for num_iters iterations.

+
w = 1e-2 * mx.random.normal((num_features,))
+
+for _ in range(num_iters):
+    grad = grad_fn(w)
+    w = w - lr * grad
+    mx.eval(w)
+
+
+

Finally, compute the loss of the learned parameters and verify that they are +close to the ground truth parameters.

+
loss = loss_fn(w)
+error_norm = mx.sum(mx.square(w - w_star)).item() ** 0.5
+
+print(
+    f"Loss {loss.item():.5f}, |w-w*| = {error_norm:.5f}, "
+)
+# Should print something close to: Loss 0.00005, |w-w*| = 0.00364
+
+
+

Complete linear regression +and logistic regression +examples are available in the MLX GitHub repo.

+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/examples/llama-inference.html b/docs/build/html/examples/llama-inference.html new file mode 100644 index 000000000..4188bfad9 --- /dev/null +++ b/docs/build/html/examples/llama-inference.html @@ -0,0 +1,514 @@ + + + + + + + LLM inference — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

LLM inference

+

MLX enables efficient inference of large-ish transformers on Apple silicon +without compromising on ease of use. In this example we will create an +inference script for the Llama family of transformer models in which the model +is defined in less than 200 lines of python.

+
+

Implementing the model

+

We will use the neural network building blocks defined in the mlx.nn +module to concisely define the model architecture.

+
+

Attention layer

+

We will start with the llama attention layer which notably uses the RoPE +positional encoding. [1] In addition, our attention layer will optionally use a +key/value cache that will be concatenated with the provided keys and values to +support efficient inference.

+

Our implementation uses mlx.nn.Linear for all the projections and +mlx.nn.RoPE for the positional encoding.

+
import mlx.core as mx
+import mlx.nn as nn
+
+class LlamaAttention(nn.Module):
+    def __init__(self, dims: int, num_heads: int):
+        super().__init__()
+
+        self.num_heads = num_heads
+
+        self.rope = nn.RoPE(dims // num_heads, traditional=True)
+        self.query_proj = nn.Linear(dims, dims, bias=False)
+        self.key_proj = nn.Linear(dims, dims, bias=False)
+        self.value_proj = nn.Linear(dims, dims, bias=False)
+        self.out_proj = nn.Linear(dims, dims, bias=False)
+
+    def __call__(self, queries, keys, values, mask=None, cache=None):
+        queries = self.query_proj(queries)
+        keys = self.key_proj(keys)
+        values = self.value_proj(values)
+
+        # Extract some shapes
+        num_heads = self.num_heads
+        B, L, D = queries.shape
+
+        # Prepare the queries, keys and values for the attention computation
+        queries = queries.reshape(B, L, num_heads, -1).transpose(0, 2, 1, 3)
+        keys = keys.reshape(B, L, num_heads, -1).transpose(0, 2, 1, 3)
+        values = values.reshape(B, L, num_heads, -1).transpose(0, 2, 1, 3)
+
+        # Add RoPE to the queries and keys and combine them with the cache
+        if cache is not None:
+            key_cache, value_cache = cache
+            queries = self.rope(queries, offset=key_cache.shape[2])
+            keys = self.rope(keys, offset=key_cache.shape[2])
+            keys = mx.concatenate([key_cache, keys], axis=2)
+            values = mx.concatenate([value_cache, values], axis=2)
+        else:
+            queries = self.rope(queries)
+            keys = self.rope(keys)
+
+        # Finally perform the attention computation
+        scale = math.sqrt(1 / queries.shape[-1])
+        scores = (queries * scale) @ keys.transpose(0, 1, 3, 2)
+        if mask is not None:
+            scores = scores + mask
+        scores = mx.softmax(scores, axis=-1)
+        values_hat = (scores @ values).transpose(0, 2, 1, 3).reshape(B, L, -1)
+
+        # Note that we return the keys and values to possibly be used as a cache
+        return self.out_proj(values_hat), (keys, values)
+
+
+
+
+

Encoder layer

+

The other component of the Llama model is the encoder layer which uses RMS +normalization [2] and SwiGLU. [3] For RMS normalization we will use +mlx.nn.RMSNorm that is already provided in mlx.nn.

+
class LlamaEncoderLayer(nn.Module):
+    def __init__(self, dims: int, mlp_dims: int, num_heads: int):
+        super().__init__()
+
+        self.attention = LlamaAttention(dims, num_heads)
+
+        self.norm1 = nn.RMSNorm(dims)
+        self.norm2 = nn.RMSNorm(dims)
+
+        self.linear1 = nn.Linear(dims, mlp_dims, bias=False)
+        self.linear2 = nn.Linear(dims, mlp_dims, bias=False)
+        self.linear3 = nn.Linear(mlp_dims, dims, bias=False)
+
+    def __call__(self, x, mask=None, cache=None):
+        y = self.norm1(x)
+        y, cache = self.attention(y, y, y, mask, cache)
+        x = x + y
+
+        y = self.norm2(x)
+        a = self.linear1(y)
+        b = self.linear2(y)
+        y = a * mx.sigmoid(a) * b
+        y = self.linear3(y)
+        x = x + y
+
+        return x, cache
+
+
+
+
+

Full model

+

To implement any Llama model we simply have to combine LlamaEncoderLayer +instances with an mlx.nn.Embedding to embed the input tokens.

+
class Llama(nn.Module):
+    def __init__(
+        self, num_layers: int, vocab_size: int, dims: int, mlp_dims: int, num_heads: int
+    ):
+        super().__init__()
+
+        self.embedding = nn.Embedding(vocab_size, dims)
+        self.layers = [
+            LlamaEncoderLayer(dims, mlp_dims, num_heads) for _ in range(num_layers)
+        ]
+        self.norm = nn.RMSNorm(dims)
+        self.out_proj = nn.Linear(dims, vocab_size, bias=False)
+
+    def __call__(self, x):
+        mask = nn.MultiHeadAttention.create_additive_causal_mask(x.shape[1])
+        mask = mask.astype(self.embedding.weight.dtype)
+
+        x = self.embedding(x)
+        for l in self.layers:
+            x, _ = l(x, mask)
+        x = self.norm(x)
+        return self.out_proj(x)
+
+
+

Note that in the implementation above we use a simple list to hold the encoder +layers but using model.parameters() will still consider these layers.

+
+
+

Generation

+

Our Llama module can be used for training but not inference as the +__call__ method above processes one input, completely ignores the cache and +performs no sampling whatsoever. In the rest of this subsection, we will +implement the inference function as a python generator that processes the +prompt and then autoregressively yields tokens one at a time.

+
class Llama(nn.Module):
+    ...
+
+    def generate(self, x, temp=1.0):
+        cache = []
+
+        # Make an additive causal mask. We will need that to process the prompt.
+        mask = nn.MultiHeadAttention.create_additive_causal_mask(x.shape[1])
+        mask = mask.astype(self.embedding.weight.dtype)
+
+        # First we process the prompt x the same way as in __call__ but
+        # save the caches in cache
+        x = self.embedding(x)
+        for l in self.layers:
+            x, c = l(x, mask=mask)
+            cache.append(c)  # <--- we store the per layer cache in a
+                             #      simple python list
+        x = self.norm(x)
+        y = self.out_proj(x[:, -1])  # <--- we only care about the last logits
+                                     #      that generate the next token
+        y = mx.random.categorical(y * (1/temp))
+
+        # y now has size [1]
+        # Since MLX is lazily evaluated nothing is computed yet.
+        # Calling y.item() would force the computation to happen at
+        # this point but we can also choose not to do that and let the
+        # user choose when to start the computation.
+        yield y
+
+        # Now we parsed the prompt and generated the first token we
+        # need to feed it back into the model and loop to generate the
+        # rest.
+        while True:
+            # Unsqueezing the last dimension to add a sequence length
+            # dimension of 1
+            x = y[:, None]
+
+            x = self.embedding(x)
+            for i in range(len(cache)):
+                # We are overwriting the arrays in the cache list. When
+                # the computation will happen, MLX will be discarding the
+                # old cache the moment it is not needed anymore.
+                x, cache[i] = self.layers[i](x, mask=None, cache=cache[i])
+            x = self.norm(x)
+            y = self.out_proj(x[:, -1])
+            y = mx.random.categorical(y * (1/temp))
+
+            yield y
+
+
+
+
+

Putting it all together

+

We now have everything we need to create a Llama model and sample tokens from +it. In the following code, we randomly initialize a small Llama model, process +6 tokens of prompt and generate 10 tokens.

+
model = Llama(num_layers=12, vocab_size=8192, dims=512, mlp_dims=1024, num_heads=8)
+
+# Since MLX is lazily evaluated nothing has actually been materialized yet.
+# We could have set the `dims` to 20_000 on a machine with 8GB of RAM and the
+# code above would still run. Let's actually materialize the model.
+mx.eval(model.parameters())
+
+prompt = mx.array([[1, 10, 8, 32, 44, 7]])  # <-- Note the double brackets because we
+                                            #     have a batch dimension even
+                                            #     though it is 1 in this case
+
+generated = [t for i, t in zip(range(10), model.generate(prompt, 0.8))]
+
+# Since we haven't evaluated anything, nothing is computed yet. The list
+# `generated` contains the arrays that hold the computation graph for the
+# full processing of the prompt and the generation of 10 tokens.
+#
+# We can evaluate them one at a time, or all together. Concatenate them or
+# print them. They would all result in very similar runtimes and give exactly
+# the same results.
+mx.eval(generated)
+
+
+
+
+
+

Converting the weights

+

This section assumes that you have access to the original Llama weights and the +SentencePiece model that comes with them. We will write a small script to +convert the PyTorch weights to MLX compatible ones and write them in a NPZ file +that can be loaded directly by MLX.

+
import argparse
+from itertools import starmap
+
+import numpy as np
+import torch
+
+def map_torch_to_mlx(key, value):
+    if "tok_embedding" in key:
+        key = "embedding.weight"
+
+    elif "norm" in key:
+        key = key.replace("attention_norm", "norm1").replace("ffn_norm", "norm2")
+
+    elif "wq" in key or "wk" in key or "wv" in key or "wo" in key:
+        key = key.replace("wq", "query_proj")
+        key = key.replace("wk", "key_proj")
+        key = key.replace("wv", "value_proj")
+        key = key.replace("wo", "out_proj")
+
+    elif "w1" in key or "w2" in key or "w3" in key:
+        # The FFN is a separate submodule in PyTorch
+        key = key.replace("feed_forward.w1", "linear1")
+        key = key.replace("feed_forward.w3", "linear2")
+        key = key.replace("feed_forward.w2", "linear3")
+
+    elif "output" in key:
+        key = key.replace("output", "out_proj")
+
+    elif "rope" in key:
+        return None, None
+
+    return key, value.numpy()
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description="Convert Llama weights to MLX")
+    parser.add_argument("torch_weights")
+    parser.add_argument("output_file")
+    args = parser.parse_args()
+
+    state = torch.load(args.torch_weights)
+    np.savez(
+        args.output_file,
+        **{k: v for k, v in starmap(map_torch_to_mlx, state.items()) if k is not None}
+    )
+
+
+
+
+

Weight loading and benchmarking

+

After converting the weights to be compatible to our implementation, all that is +left is to load them from disk and we can finally use the LLM to generate text. +We can load numpy format files using the mlx.core.load() operation.

+

To create a parameter dictionary from the key/value representation of NPZ files +we will use the mlx.utils.tree_unflatten() helper method as follows:

+
from mlx.utils import tree_unflatten
+
+model.update(tree_unflatten(list(mx.load(weight_file).items())))
+
+
+

mlx.utils.tree_unflatten() will take keys from the NPZ file that look +like layers.2.attention.query_proj.weight and will transform them to

+
{"layers": [..., ..., {"attention": {"query_proj": {"weight": ...}}}]}
+
+
+

which can then be used to update the model. Note that the method above incurs +several unnecessary copies from disk to numpy and then from numpy to MLX. It +will be replaced in the future with direct loading to MLX.

+

You can download the full example code in mlx-examples. Assuming, the +existence of weights.pth and tokenizer.model in the current working +directory we can play around with our inference script as follows (the timings +are representative of an M1 Ultra and the 7B parameter Llama model):

+
$ python convert.py weights.pth llama-7B.mlx.npz
+$ python llama.py llama-7B.mlx.npz tokenizer.model 'Call me Ishmael. Some years ago never mind how long precisely'
+[INFO] Loading model from disk: 5.247 s
+Press enter to start generation
+------
+, having little or no money in my purse, and nothing of greater consequence in my mind, I happened to be walking down Gower Street in the afternoon, in the heavy rain, and I saw a few steps off, a man in rags, who sat upon his bundle and looked hard into the wet as if he were going to cry. I watched him attentively for some time, and could not but observe that, though a numerous crowd was hurrying up and down,
+------
+[INFO] Prompt processing: 0.437 s
+[INFO] Full generation: 4.330 s
+
+
+

We observe that 4.3 seconds are required to generate 100 tokens and 0.4 seconds +of those are spent processing the prompt. This amounts to a little over 39 ms +per token.

+

By running with a much bigger prompt we can see that the per token generation +time as well as the prompt processing time remains almost constant.

+
$ python llama.py llama-7B.mlx.npz tokenizer.model 'Call me Ishmael. Some years ago never mind how long precisely, having little or no money in my purse, and nothing of greater consequence in my mind, I happened to be walking down Gower Street in the afternoon, in the heavy rain, and I saw a few steps off, a man in rags, who sat upon his bundle and looked hard into the wet as if he were going to cry. I watched him attentively for some time, and could not but observe that, though a numerous crowd was hurrying up and down, nobody took the least notice of him. I stopped at last, at a little distance, as if I had been in doubt, and after looking on a few minutes, walked straight up to him. He slowly raised his eyes, and fixed them upon me for a moment, without speaking, and then resumed his place and posture as before. I stood looking at him for a while, feeling very much pain at heart, and then said to him, “What are you doing there?” Something like a smile passed over his face, as he said slowly, “I am waiting for someone; but it has been three quarters of an hour now, and he has not come.” “What is it you are waiting for?” said I. Still he made no immediate reply, but again put his face down upon his hands, and did not'
+[INFO] Loading model from disk: 5.247 s
+Press enter to start generation
+------
+take his eyes from the ground. “What is it you are waiting for?” said I. “I am not accustomed to be thus questioned,” said he. “You look like a reasonable man—tell me, then, what are you waiting for?” “You would not understand,” he replied; “and how could you help me, if I were to tell you?” “I should not only understand, but would do all that I could,” said I. He did not
+------
+[INFO] Prompt processing: 0.579 s
+[INFO] Full generation: 4.690 s
+$ python llama.py --num-tokens 500 llama-7B.mlx.npz tokenizer.model 'Call me Ishmael. Some years ago never mind how long precisely, having little or no money in my purse, and nothing of greater consequence in my mind, I happened to be walking down Gower Street in the afternoon, in the heavy rain, and I saw a few steps off, a man in rags, who sat upon his bundle and looked hard into the wet as if he were going to cry. I watched him attentively for some time, and could not but observe that, though a numerous crowd was hurrying up and down, nobody took the least notice of him. I stopped at last, at a little distance, as if I had been in doubt, and after looking on a few minutes, walked straight up to him. He slowly raised his eyes, and fixed them upon me for a moment, without speaking, and then resumed his place and posture as before. I stood looking at him for a while, feeling very much pain at heart, and then said to him, “What are you doing there?” Something like a smile passed over his face, as he said slowly, “I am waiting for someone; but it has been three quarters of an hour now, and he has not come.” “What is it you are waiting for?” said I. Still he made no immediate reply, but again put his face down upon his hands, and did not'
+[INFO] Loading model from disk: 5.628 s
+Press enter to start generation
+------
+take his eyes from the ground. “What is it you are waiting for?” said I. “I am not accustomed to be thus questioned,” said he. “You look like a reasonable man—tell me, then, what are you waiting for?” “You would not understand,” he replied; “and how could you help me, if I were to tell you?” “I should not only understand, but would do all that I could,” said I. He did not reply, but still went on looking at the ground, and took hold of his bundle with a nervous trembling. I waited some time, and then resumed. “It is of no use to say you would not understand, if I were to tell you,” said he. “I have not told you why I am waiting for him,” said I. “And I am sure I should not understand,” replied he. “I will tell you then,” said I, “and, perhaps, you would not be surprised.” “No matter,” said he, “I shall be surprised anyhow; so tell me why you are waiting for him.” “He is my friend,” said I. “Yes,” said he, with a slight smile, “I know.” “He has been kind to me,” said I, “and I am waiting for him. I want to see him, and could have waited as I am now, for a much longer time.” “He will not soon come,” said he. “Unless he sees you here, he will not know of your having waited, and he will be very unlikely to come.” “No matter,” said I, “I shall wait for him.” “This is a strange thing,” said he, still with the same amused smile. “How did you know,” said I, “that he was coming? How should you be waiting?” “That is my secret,” said he. “And you expect him?” “Yes,” said I. “Are you disappointed then, if he does not come?” “No,” said I, “it is his secret, not mine.” “If he comes,” said he, “do you mean to go straight away?” “Yes,” said I, “I cannot be happy if I do not go straight away after him.” “Did you know this place before?” asked he. “Yes,” said I. “Is there any shop to buy food here?” “
+------
+[INFO] Prompt processing: 0.633 s
+[INFO] Full generation: 21.475 s
+
+
+
+
+

Scripts

+
+

Download the code

+

The full example code is available in mlx-examples.

+
+ + + +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/examples/mlp.html b/docs/build/html/examples/mlp.html new file mode 100644 index 000000000..801e05c8e --- /dev/null +++ b/docs/build/html/examples/mlp.html @@ -0,0 +1,252 @@ + + + + + + + Multi-Layer Perceptron — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Multi-Layer Perceptron

+

In this example we’ll learn to use mlx.nn by implementing a simple +multi-layer perceptron to classify MNIST.

+

As a first step import the MLX packages we need:

+
import mlx.core as mx
+import mlx.nn as nn
+import mlx.optimizers as optim
+
+import numpy as np
+
+
+

The model is defined as the MLP class which inherits from +mlx.nn.Module. We follow the standard idiom to make a new module:

+
    +

  1. Define an __init__ where the parameters and/or submodules are setup. See +the Module class docs for more information on how +mlx.nn.Module registers parameters.

    2. +

  2. Define a __call__ where the computation is implemented.

    3. +
+
class MLP(nn.Module):
+    def __init__(
+        self, num_layers: int, input_dim: int, hidden_dim: int, output_dim: int
+    ):
+        super().__init__()
+        layer_sizes = [input_dim] + [hidden_dim] * num_layers + [output_dim]
+        self.layers = [
+            nn.Linear(idim, odim)
+            for idim, odim in zip(layer_sizes[:-1], layer_sizes[1:])
+        ]
+
+    def __call__(self, x):
+        for l in self.layers[:-1]:
+            x = mx.maximum(l(x), 0.0)
+        return self.layers[-1](x)
+
+
+

We define the loss function which takes the mean of the per-example cross +entropy loss. The mlx.nn.losses sub-package has implementations of some +commonly used loss functions.

+
def loss_fn(model, X, y):
+    return mx.mean(nn.losses.cross_entropy(model(X), y))
+
+
+

We also need a function to compute the accuracy of the model on the validation +set:

+
def eval_fn(model, X, y):
+    return mx.mean(mx.argmax(model(X), axis=1) == y)
+
+
+

Next, setup the problem parameters and load the data:

+
num_layers = 2
+hidden_dim = 32
+num_classes = 10
+batch_size = 256
+num_epochs = 10
+learning_rate = 1e-1
+
+# Load the data
+import mnist
+train_images, train_labels, test_images, test_labels = map(
+    mx.array, mnist.mnist()
+)
+
+
+

Since we’re using SGD, we need an iterator which shuffles and constructs +minibatches of examples in the training set:

+
def batch_iterate(batch_size, X, y):
+    perm = mx.array(np.random.permutation(y.size))
+    for s in range(0, y.size, batch_size):
+        ids = perm[s : s + batch_size]
+        yield X[ids], y[ids]
+
+
+

Finally, we put it all together by instantiating the model, the +mlx.optimizers.SGD optimizer, and running the training loop:

+
# Load the model
+model = MLP(num_layers, train_images.shape[-1], hidden_dim, num_classes)
+mx.eval(model.parameters())
+
+# Get a function which gives the loss and gradient of the
+# loss with respect to the model's trainable parameters
+loss_and_grad_fn = nn.value_and_grad(model, loss_fn)
+
+# Instantiate the optimizer
+optimizer = optim.SGD(learning_rate=learning_rate)
+
+for e in range(num_epochs):
+    for X, y in batch_iterate(batch_size, train_images, train_labels):
+        loss, grads = loss_and_grad_fn(model, X, y)
+
+        # Update the optimizer state and model parameters
+        # in a single call
+        optimizer.update(model, grads)
+
+        # Force a graph evaluation
+        mx.eval(model.parameters(), optimizer.state)
+
+    accuracy = eval_fn(model, test_images, test_labels)
+    print(f"Epoch {e}: Test accuracy {accuracy.item():.3f}")
+
+
+
+

Note

+

The mlx.nn.value_and_grad() function is a convenience function to get +the gradient of a loss with respect to the trainable parameters of a model. +This should not be confused with mlx.core.value_and_grad().

+
+

The model should train to a decent accuracy (about 95%) after just a few passes +over the training set. The full example +is available in the MLX GitHub repo.

+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/genindex.html b/docs/build/html/genindex.html new file mode 100644 index 000000000..2fe4c4d4f --- /dev/null +++ b/docs/build/html/genindex.html @@ -0,0 +1,788 @@ + + + + + + Index — MLX 0.0.0 documentation + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+
    +
    • +
  • Index
    • +
  • +
    • +
+
+
+
+
+ + +

Index

+ +
+ _ + | A + | B + | C + | D + | E + | F + | G + | I + | J + | K + | L + | M + | N + | O + | P + | R + | S + | T + | U + | V + | W + | Z + +
+

_

+ + +
+ +

A

+ + + +
+ +

B

+ + + +
+ +

C

+ + + +
+ +

D

+ + + +
+ +

E

+ + + +
+ +

F

+ + + +
+ +

G

+ + + +
+ +

I

+ + + +
+ +

J

+ + +
+ +

K

+ + +
+ +

L

+ + + +
+ +

M

+ + + +
+ +

N

+ + + +
+ +

O

+ + + +
+ +

P

+ + + +
+ +

R

+ + + +
+ +

S

+ + + +
+ +

T

+ + + +
+ +

U

+ + + +
+ +

V

+ + + +
+ +

W

+ + +
+ +

Z

+ + + +
+ + + +
+
+
+ +
+ +
+

© Copyright 2023, MLX Contributors.

+
+ + Built with Sphinx using a + theme + provided by Read the Docs. + + +
+
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/index.html b/docs/build/html/index.html new file mode 100644 index 000000000..0b896db8a --- /dev/null +++ b/docs/build/html/index.html @@ -0,0 +1,188 @@ + + + + + + + MLX — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

MLX

+
+

Install

+ +
+
+

Usage

+ +
+
+

Examples

+ +
+
+

Further Reading

+ +
+
+

Python API Reference

+ +
+
+

C++ API Reference

+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/install.html b/docs/build/html/install.html new file mode 100644 index 000000000..c880d6e80 --- /dev/null +++ b/docs/build/html/install.html @@ -0,0 +1,244 @@ + + + + + + + Build and Install — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Build and Install

+
+

Install from PyPI

+

MLX is available at Apple’s internal PyPI repository. All you have to do to use +MLX with your own Apple silicon computer is

+
pip install apple-mlx -i https://pypi.apple.com/simple
+
+
+
+
+

Build from source

+
+

Build Requirements

+
    +

  • A C++ compiler with C++17 support (e.g. Clang >= 5.0)

    • +

  • cmake – version 3.24 or later, and make

    • +
+
+
+

Python API

+

To build and install the MLX python library from source, first, clone MLX from +its GitHub repo:

+
git clone git@github.com:ml-explore/mlx.git mlx && cd mlx
+
+
+

Make sure that you have pybind11 +installed. You can install pybind11 with pip, brew or conda as follows:

+
pip install "pybind11[global]"
+conda install pybind11
+brew install pybind11
+
+
+

Then simply build and install it using pip:

+
env CMAKE_BUILD_PARALLEL_LEVEL="" pip install .
+
+
+
+
+

C++ API

+

Currently, MLX must be built and installed from source.

+

Similarly to the python library, to build and install the MLX C++ library start +by cloning MLX from its GitHub repo:

+
git clone git@github.com:ml-explore/mlx.git mlx && cd mlx
+
+
+

Create a build directory and run CMake and make:

+
mkdir -p build && cd build
+cmake .. && make -j
+
+
+

Run tests with:

+
make test
+
+
+

Install with:

+
make install
+
+
+

Note that the built mlx.metallib file should be either at the same +directory as the executable statically linked to libmlx.a or the +preprocessor constant METAL_PATH should be defined at build time and it +should point to the path to the built metal library.

+ + ++++ + + + + + + + + + + + + + + + + + + + + + + +
Build Options

Option

Default

MLX_BUILD_TESTS

ON

MLX_BUILD_EXAMPLES

OFF

MLX_BUILD_BENCHMARKS

OFF

MLX_BUILD_METAL

ON

MLX_BUILD_PYTHON_BINDINGS

OFF

+
+
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/objects.inv b/docs/build/html/objects.inv new file mode 100644 index 000000000..a5f5735d5 Binary files /dev/null and b/docs/build/html/objects.inv differ diff --git a/docs/build/html/python/_autosummary/mlx.core.Device.html b/docs/build/html/python/_autosummary/mlx.core.Device.html new file mode 100644 index 000000000..3d9bfbece --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.Device.html @@ -0,0 +1,179 @@ + + + + + + + mlx.core.Device — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.Device

+
+
+class mlx.core.Device
+
+
+__init__(self: Device, type: mlx.core.DeviceType, index: int = 0) → None
+
+ +

Methods

+ + + + + + +

__init__(self, type[, index])

+

Attributes

+ + + + + + +

type

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.Dtype.html b/docs/build/html/python/_autosummary/mlx.core.Dtype.html new file mode 100644 index 000000000..3794a85f0 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.Dtype.html @@ -0,0 +1,210 @@ + + + + + + + mlx.core.Dtype — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.Dtype

+
+
+class mlx.core.Dtype
+

An object to hold the type of a array.

+

See the list of types for more details +on available data types.

+
+
+__init__(*args, **kwargs)
+
+ +

Methods

+ + + + + + +

__init__(*args, **kwargs)

+

Attributes

+ + + + + + +

size

Size of the type in bytes.

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.Stream.html b/docs/build/html/python/_autosummary/mlx.core.Stream.html new file mode 100644 index 000000000..dd0be74d5 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.Stream.html @@ -0,0 +1,179 @@ + + + + + + + mlx.core.Stream — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.Stream

+
+
+class mlx.core.Stream
+
+
+__init__(self: Stream, index: int, device: Device) → None
+
+ +

Methods

+ + + + + + +

__init__(self, index, device)

+

Attributes

+ + + + + + +

device

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.abs.html b/docs/build/html/python/_autosummary/mlx.core.abs.html new file mode 100644 index 000000000..fe939efb1 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.abs.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.abs — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.abs

+
+
+mlx.core.abs(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise absolute value.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The absolute value of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.add.html b/docs/build/html/python/_autosummary/mlx.core.add.html new file mode 100644 index 000000000..ec0fcbfd9 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.add.html @@ -0,0 +1,252 @@ + + + + + + + mlx.core.add — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.add

+
+
+mlx.core.add(a: Union[bool, int, float, complex, array], b: Union[bool, int, float, complex, array], /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise addition.

+

Add two arrays with numpy-style broadcasting semantics. Either or both input arrays +can also be scalars.

+
+
Parameters:
+
    +

  • a (array) – Input array or scalar.

    • +

  • b (array) – Input array or scalar.

    • +
+
+
Returns:
+

The sum of a and b.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.all.html b/docs/build/html/python/_autosummary/mlx.core.all.html new file mode 100644 index 000000000..348831405 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.all.html @@ -0,0 +1,254 @@ + + + + + + + mlx.core.all — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.all

+
+
+mlx.core.all(a: array, /, axis: Union[None, int, List[int]] = None, keepdims: bool = False, *, stream: Union[None, Stream, Device] = None) → array
+

An and reduction over the given axes.

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • axis (int or list(int), optional) – Optional axis or +axes to reduce over. If unspecified this defaults +to reducing over the entire array.

    • +

  • keepdims (bool, optional) – Keep reduced axes as +singleton dimensions, defaults to False.

    • +
+
+
Returns:
+

The output array with the corresponding axes reduced.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.allclose.html b/docs/build/html/python/_autosummary/mlx.core.allclose.html new file mode 100644 index 000000000..347148610 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.allclose.html @@ -0,0 +1,258 @@ + + + + + + + mlx.core.allclose — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.allclose

+
+
+mlx.core.allclose(a: array, b: array, /, rtol: float = 1e-05, atol: float = 1e-08, *, stream: Union[None, Stream, Device] = None) → array
+

Approximate comparison of two arrays.

+

The arrays are considered equal if:

+
all(abs(a - b) <= (atol + rtol * abs(b)))
+
+
+

Note unlike array_equal(), this function supports numpy-style +broadcasting.

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • b (array) – Input array.

    • +

  • rtol (float) – Relative tolerance.

    • +

  • atol (float) – Absolute tolerance.

    • +
+
+
Returns:
+

The boolean output scalar indicating if the arrays are close.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.any.html b/docs/build/html/python/_autosummary/mlx.core.any.html new file mode 100644 index 000000000..5633452bf --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.any.html @@ -0,0 +1,254 @@ + + + + + + + mlx.core.any — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.any

+
+
+mlx.core.any(a: array, /, axis: Union[None, int, List[int]] = None, keepdims: bool = False, *, stream: Union[None, Stream, Device] = None) → array
+

An or reduction over the given axes.

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • axis (int or list(int), optional) – Optional axis or +axes to reduce over. If unspecified this defaults +to reducing over the entire array.

    • +

  • keepdims (bool, optional) – Keep reduced axes as +singleton dimensions, defaults to False.

    • +
+
+
Returns:
+

The output array with the corresponding axes reduced.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.arange.html b/docs/build/html/python/_autosummary/mlx.core.arange.html new file mode 100644 index 000000000..e487ef578 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.arange.html @@ -0,0 +1,264 @@ + + + + + + + mlx.core.arange — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.arange

+
+
+mlx.core.arange(start, stop, step, dtype: Optional[Dtype] = None, *, stream: Union[None, Stream, Device] = None) → array
+

Generates ranges of numbers.

+

Generate numbers in the half-open interval [start, stop) in +increments of step.

+
+
Parameters:
+
    +

  • start (float or int, optional) – Starting value which defaults to 0.

    • +

  • stop (float or int) – Stopping value.

    • +

  • step (float or int, optional) – Increment which defaults to 1.

    • +

  • dtype (Dtype, optional) – Specifies the data type of the output. +If unspecified will default to float32 if any of start, +stop, or step are float. Otherwise will default to +int32.

    • +
+
+
Returns:
+

The range of values.

+
+
Return type:
+

array

+
+
+
+

Note

+

Following the Numpy convention the actual increment used to +generate numbers is dtype(start + step) - dtype(start). +This can lead to unexpected results for example if start + step +is a fractional value and the dtype is integral.

+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.arccos.html b/docs/build/html/python/_autosummary/mlx.core.arccos.html new file mode 100644 index 000000000..d89a55c33 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.arccos.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.arccos — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.arccos

+
+
+mlx.core.arccos(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise inverse cosine.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The inverse cosine of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.arccosh.html b/docs/build/html/python/_autosummary/mlx.core.arccosh.html new file mode 100644 index 000000000..50b744bc6 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.arccosh.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.arccosh — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.arccosh

+
+
+mlx.core.arccosh(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise inverse hyperbolic cosine.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The inverse hyperbolic cosine of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.arcsin.html b/docs/build/html/python/_autosummary/mlx.core.arcsin.html new file mode 100644 index 000000000..19a1c7f75 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.arcsin.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.arcsin — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.arcsin

+
+
+mlx.core.arcsin(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise inverse sine.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The inverse sine of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.arcsinh.html b/docs/build/html/python/_autosummary/mlx.core.arcsinh.html new file mode 100644 index 000000000..02464e217 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.arcsinh.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.arcsinh — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.arcsinh

+
+
+mlx.core.arcsinh(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise inverse hyperbolic sine.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The inverse hyperbolic sine of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.arctan.html b/docs/build/html/python/_autosummary/mlx.core.arctan.html new file mode 100644 index 000000000..a7291e994 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.arctan.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.arctan — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.arctan

+
+
+mlx.core.arctan(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise inverse tangent.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The inverse tangent of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.arctanh.html b/docs/build/html/python/_autosummary/mlx.core.arctanh.html new file mode 100644 index 000000000..861d5475b --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.arctanh.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.arctanh — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.arctanh

+
+
+mlx.core.arctanh(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise inverse hyperbolic tangent.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The inverse hyperbolic tangent of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.argmax.html b/docs/build/html/python/_autosummary/mlx.core.argmax.html new file mode 100644 index 000000000..03aa5babc --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.argmax.html @@ -0,0 +1,253 @@ + + + + + + + mlx.core.argmax — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.argmax

+
+
+mlx.core.argmax(a: array, /, axis: Optional[int] = None, keepdims: bool = False, *, stream: Union[None, Stream, Device] = None) → array
+

Indices of the maximum values along the axis.

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • axis (int, optional) – Optional axis to reduce over. If unspecified +this defaults to reducing over the entire array.

    • +

  • keepdims (bool, optional) – Keep reduced axes as +singleton dimensions, defaults to False.

    • +
+
+
Returns:
+

The output array with the indices of the minimum values.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.argmin.html b/docs/build/html/python/_autosummary/mlx.core.argmin.html new file mode 100644 index 000000000..bf3687e20 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.argmin.html @@ -0,0 +1,253 @@ + + + + + + + mlx.core.argmin — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.argmin

+
+
+mlx.core.argmin(a: array, /, axis: Optional[int] = None, keepdims: bool = False, *, stream: Union[None, Stream, Device] = None) → array
+

Indices of the minimum values along the axis.

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • axis (int, optional) – Optional axis to reduce over. If unspecified +this defaults to reducing over the entire array.

    • +

  • keepdims (bool, optional) – Keep reduced axes as +singleton dimensions, defaults to False.

    • +
+
+
Returns:
+

The output array with the indices of the minimum values.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.argpartition.html b/docs/build/html/python/_autosummary/mlx.core.argpartition.html new file mode 100644 index 000000000..c0e32895f --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.argpartition.html @@ -0,0 +1,259 @@ + + + + + + + mlx.core.argpartition — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.argpartition

+
+
+mlx.core.argpartition(a: array, /, kth: int, axis: Optional[int] = - 1, *, stream: Union[None, Stream, Device] = None) → array
+

Returns the indices that partition the array.

+

The ordering of the elements within a partition in given by the indices +is undefined.

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • kth (int) – Element index at the kth position in the output will +give the sorted position. All indices before the kth position +will be of elements less or equal to the element at the kth +index and all indices after will be of elements greater or equal +to the element at the kth index.

    • +

  • axis (int or None, optional) – Optional axis to partiton over. +If None, this partitions over the flattened array. +If unspecified, it defaults to -1.

    • +
+
+
Returns:
+

The indices that partition the input array.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.argsort.html b/docs/build/html/python/_autosummary/mlx.core.argsort.html new file mode 100644 index 000000000..fca68769f --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.argsort.html @@ -0,0 +1,252 @@ + + + + + + + mlx.core.argsort — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.argsort

+
+
+mlx.core.argsort(a: array, /, axis: Optional[int] = - 1, *, stream: Union[None, Stream, Device] = None) → array
+

Returns the indices that sort the array.

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • axis (int or None, optional) – Optional axis to sort over. +If None, this sorts over the flattened array. +If unspecified, it defaults to -1 (sorting over the last axis).

    • +
+
+
Returns:
+

The indices that sort the input array.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.T.html b/docs/build/html/python/_autosummary/mlx.core.array.T.html new file mode 100644 index 000000000..49798f462 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.T.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.T — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.T

+
+
+property array.T
+

Equivalent to calling self.transpose() with no arguments.

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.abs.html b/docs/build/html/python/_autosummary/mlx.core.array.abs.html new file mode 100644 index 000000000..242e5f21a --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.abs.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.abs — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.abs

+
+
+array.abs(self: array, *, stream: Union[None, Stream, Device] = None) → array
+

See abs().

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.all.html b/docs/build/html/python/_autosummary/mlx.core.array.all.html new file mode 100644 index 000000000..643cd07a5 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.all.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.all — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.all

+
+
+array.all(self: array, axis: Union[None, int, List[int]] = None, keepdims: bool = False, *, stream: Union[None, Stream, Device] = None) → array
+

See all().

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.any.html b/docs/build/html/python/_autosummary/mlx.core.array.any.html new file mode 100644 index 000000000..0ea314126 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.any.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.any — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.any

+
+
+array.any(self: array, axis: Union[None, int, List[int]] = None, keepdims: bool = False, *, stream: Union[None, Stream, Device] = None) → array
+

See any().

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.argmax.html b/docs/build/html/python/_autosummary/mlx.core.array.argmax.html new file mode 100644 index 000000000..01311ba58 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.argmax.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.argmax — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.argmax

+
+
+array.argmax(self: array, axis: Optional[int] = None, keepdims: bool = False, *, stream: Union[None, Stream, Device] = None) → array
+

See argmax().

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.argmin.html b/docs/build/html/python/_autosummary/mlx.core.array.argmin.html new file mode 100644 index 000000000..90a9e5288 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.argmin.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.argmin — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.argmin

+
+
+array.argmin(self: array, axis: Optional[int] = None, keepdims: bool = False, *, stream: Union[None, Stream, Device] = None) → array
+

See argmin().

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.astype.html b/docs/build/html/python/_autosummary/mlx.core.array.astype.html new file mode 100644 index 000000000..288ac7dfc --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.astype.html @@ -0,0 +1,201 @@ + + + + + + + mlx.core.array.astype — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.astype

+
+
+array.astype(self: array, dtype: Dtype, stream: Union[None, Stream, Device] = None) → array
+

Cast the array to a specified type.

+
+
Parameters:
+
    +

  • dtype (Dtype) – Type to which the array is cast.

    • +

  • stream (Stream) – Stream (or device) for the operation.

    • +
+
+
Returns:
+

The array with type dtype.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.cos.html b/docs/build/html/python/_autosummary/mlx.core.array.cos.html new file mode 100644 index 000000000..bdc751cc3 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.cos.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.cos — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.cos

+
+
+array.cos(self: array, *, stream: Union[None, Stream, Device] = None) → array
+

See cos().

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.dtype.html b/docs/build/html/python/_autosummary/mlx.core.array.dtype.html new file mode 100644 index 000000000..f5073977f --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.dtype.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.dtype — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.dtype

+
+
+property array.dtype
+

The array’s Dtype.

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.exp.html b/docs/build/html/python/_autosummary/mlx.core.array.exp.html new file mode 100644 index 000000000..441b0f98a --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.exp.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.exp — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.exp

+
+
+array.exp(self: array, *, stream: Union[None, Stream, Device] = None) → array
+

See exp().

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.html b/docs/build/html/python/_autosummary/mlx.core.array.html new file mode 100644 index 000000000..29c254b38 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.html @@ -0,0 +1,329 @@ + + + + + + + mlx.core.array — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array

+
+
+class mlx.core.array
+

An N-dimensional array object.

+
+
+__init__(*args, **kwargs)
+

Overloaded function.

+
    +

  1. __init__(self: mlx.core.array, val: Union[bool, int, float, complex, mlx.core.array], dtype: Optional[mlx.core.Dtype] = None) -> None

    2. +

  2. __init__(self: mlx.core.array, vals: Union[list, tuple], dtype: Optional[mlx.core.Dtype] = None) -> None

    3. +

  3. __init__(self: mlx.core.array, vals: numpy.ndarray, dtype: Optional[mlx.core.Dtype] = None) -> None

    4. +

  4. __init__(self: mlx.core.array, vals: buffer, dtype: Optional[mlx.core.Dtype] = None) -> None

    5. +
+
+ +

Methods

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

__init__(*args, **kwargs)

Overloaded function.

abs(self, *[, stream])

See abs().

all(self[, axis, keepdims, stream])

See all().

any(self[, axis, keepdims, stream])

See any().

argmax(self[, axis, keepdims, stream])

See argmax().

argmin(self[, axis, keepdims, stream])

See argmin().

astype(self, dtype[, stream])

Cast the array to a specified type.

cos(self, *[, stream])

See cos().

cummax(self[, axis, reverse, inclusive, stream])

See cummax().

cummin(self[, axis, reverse, inclusive, stream])

See cummin().

cumprod(self[, axis, reverse, inclusive, stream])

See cumprod().

cumsum(self[, axis, reverse, inclusive, stream])

See cumsum().

exp(self, *[, stream])

See exp().

item(self)

Access the value of a scalar array.

log(self, *[, stream])

See log().

log10(self, *[, stream])

See log10().

log1p(self, *[, stream])

See log1p().

log2(self, *[, stream])

See log2().

logsumexp(self[, axis, keepdims, stream])

See logsumexp().

max(self[, axis, keepdims, stream])

See max().

mean(self[, axis, keepdims, stream])

See mean().

min(self[, axis, keepdims, stream])

See min().

prod(self[, axis, keepdims, stream])

See prod().

reciprocal(self, *[, stream])

See reciprocal().

reshape(self, *args[, stream])

Equivalent to reshape() but the shape can be passed either as a tuple or as separate arguments.

rsqrt(self, *[, stream])

See rsqrt().

sin(self, *[, stream])

See sin().

split(self, indices_or_sections[, axis, stream])

See split().

sqrt(self, *[, stream])

See sqrt().

square(self, *[, stream])

See square().

squeeze(self[, axis, stream])

See squeeze().

sum(self[, axis, keepdims, stream])

See sum().

tolist(self)

Convert the array to a Python list.

transpose(self, *args[, stream])

Equivalent to transpose() but the axes can be passed either as a tuple or as separate arguments.

var(self[, axis, keepdims, ddof, stream])

See var().

+

Attributes

+ + + + + + + + + + + + + + + + + + +

T

Equivalent to calling self.transpose() with no arguments.

dtype

The array's Dtype.

ndim

The array's dimension.

shape

The shape of the array as a Python list.

size

Number of elments in the array.

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.item.html b/docs/build/html/python/_autosummary/mlx.core.array.item.html new file mode 100644 index 000000000..3849b0d69 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.item.html @@ -0,0 +1,192 @@ + + + + + + + mlx.core.array.item — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.item

+
+
+array.item(self: array) → object
+

Access the value of a scalar array.

+
+
Returns:
+

Standard Python scalar.

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.log.html b/docs/build/html/python/_autosummary/mlx.core.array.log.html new file mode 100644 index 000000000..6b7a7d373 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.log.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.log — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.log

+
+
+array.log(self: array, *, stream: Union[None, Stream, Device] = None) → array
+

See log().

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.log1p.html b/docs/build/html/python/_autosummary/mlx.core.array.log1p.html new file mode 100644 index 000000000..6241ecd2b --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.log1p.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.log1p — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.log1p

+
+
+array.log1p(self: array, *, stream: Union[None, Stream, Device] = None) → array
+

See log1p().

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.logsumexp.html b/docs/build/html/python/_autosummary/mlx.core.array.logsumexp.html new file mode 100644 index 000000000..d02690c88 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.logsumexp.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.logsumexp — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.logsumexp

+
+
+array.logsumexp(self: array, axis: Union[None, int, List[int]] = None, keepdims: bool = False, *, stream: Union[None, Stream, Device] = None) → array
+

See logsumexp().

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.max.html b/docs/build/html/python/_autosummary/mlx.core.array.max.html new file mode 100644 index 000000000..a0dcee2d5 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.max.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.max — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.max

+
+
+array.max(self: array, axis: Union[None, int, List[int]] = None, keepdims: bool = False, *, stream: Union[None, Stream, Device] = None) → array
+

See max().

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.mean.html b/docs/build/html/python/_autosummary/mlx.core.array.mean.html new file mode 100644 index 000000000..e31efce50 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.mean.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.mean — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.mean

+
+
+array.mean(self: array, axis: Union[None, int, List[int]] = None, keepdims: bool = False, *, stream: Union[None, Stream, Device] = None) → array
+

See mean().

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.min.html b/docs/build/html/python/_autosummary/mlx.core.array.min.html new file mode 100644 index 000000000..3ed21d18b --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.min.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.min — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.min

+
+
+array.min(self: array, axis: Union[None, int, List[int]] = None, keepdims: bool = False, *, stream: Union[None, Stream, Device] = None) → array
+

See min().

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.ndim.html b/docs/build/html/python/_autosummary/mlx.core.array.ndim.html new file mode 100644 index 000000000..70b337467 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.ndim.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.ndim — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.ndim

+
+
+property array.ndim
+

The array’s dimension.

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.prod.html b/docs/build/html/python/_autosummary/mlx.core.array.prod.html new file mode 100644 index 000000000..240223600 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.prod.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.prod — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.prod

+
+
+array.prod(self: array, axis: Union[None, int, List[int]] = None, keepdims: bool = False, *, stream: Union[None, Stream, Device] = None) → array
+

See prod().

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.reciprocal.html b/docs/build/html/python/_autosummary/mlx.core.array.reciprocal.html new file mode 100644 index 000000000..cb8172be4 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.reciprocal.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.reciprocal — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.reciprocal

+
+
+array.reciprocal(self: array, *, stream: Union[None, Stream, Device] = None) → array
+

See reciprocal().

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.reshape.html b/docs/build/html/python/_autosummary/mlx.core.array.reshape.html new file mode 100644 index 000000000..6de8390f6 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.reshape.html @@ -0,0 +1,189 @@ + + + + + + + mlx.core.array.reshape — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.reshape

+
+
+array.reshape(self: array, *args, stream: Union[None, Stream, Device] = None) → array
+

Equivalent to reshape() but the shape can be passed either as a +tuple or as separate arguments.

+

See reshape() for full documentation.

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.rsqrt.html b/docs/build/html/python/_autosummary/mlx.core.array.rsqrt.html new file mode 100644 index 000000000..29b45ba99 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.rsqrt.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.rsqrt — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.rsqrt

+
+
+array.rsqrt(self: array, *, stream: Union[None, Stream, Device] = None) → array
+

See rsqrt().

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.shape.html b/docs/build/html/python/_autosummary/mlx.core.array.shape.html new file mode 100644 index 000000000..12e4b14ca --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.shape.html @@ -0,0 +1,195 @@ + + + + + + + mlx.core.array.shape — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.shape

+
+
+property array.shape
+

The shape of the array as a Python list.

+
+
Returns:
+

A list containing the sizes of each dimension.

+
+
Return type:
+

list(int)

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.sin.html b/docs/build/html/python/_autosummary/mlx.core.array.sin.html new file mode 100644 index 000000000..1c37e4ca6 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.sin.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.sin — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.sin

+
+
+array.sin(self: array, *, stream: Union[None, Stream, Device] = None) → array
+

See sin().

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.size.html b/docs/build/html/python/_autosummary/mlx.core.array.size.html new file mode 100644 index 000000000..ec2d259a3 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.size.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.size — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.size

+
+
+property array.size
+

Number of elments in the array.

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.split.html b/docs/build/html/python/_autosummary/mlx.core.array.split.html new file mode 100644 index 000000000..7d561db3c --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.split.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.split — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.split

+
+
+array.split(self: array, indices_or_sections: Union[int, List[int]], axis: int = 0, *, stream: Union[None, Stream, Device] = None) → List[array]
+

See split().

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.sqrt.html b/docs/build/html/python/_autosummary/mlx.core.array.sqrt.html new file mode 100644 index 000000000..b7a1d1649 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.sqrt.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.sqrt — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.sqrt

+
+
+array.sqrt(self: array, *, stream: Union[None, Stream, Device] = None) → array
+

See sqrt().

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.square.html b/docs/build/html/python/_autosummary/mlx.core.array.square.html new file mode 100644 index 000000000..1e2667f04 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.square.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.square — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.square

+
+
+array.square(self: array, *, stream: Union[None, Stream, Device] = None) → array
+

See square().

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.sum.html b/docs/build/html/python/_autosummary/mlx.core.array.sum.html new file mode 100644 index 000000000..87cde43b7 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.sum.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.sum — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.sum

+
+
+array.sum(self: array, axis: Union[None, int, List[int]] = None, keepdims: bool = False, *, stream: Union[None, Stream, Device] = None) → array
+

See sum().

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.tolist.html b/docs/build/html/python/_autosummary/mlx.core.array.tolist.html new file mode 100644 index 000000000..ed22ce5b4 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.tolist.html @@ -0,0 +1,201 @@ + + + + + + + mlx.core.array.tolist — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.tolist

+
+
+array.tolist(self: array) → object
+

Convert the array to a Python list.

+
+
Returns:
+

The Python list.

+

If the array is a scalar then a standard Python scalar is returned.

+

If the array has more than one dimension then the result is a nested +list of lists.

+

The value type of the list correpsonding to the last dimension is either +bool, int or float depending on the dtype of the array.

+

+
+
Return type:
+

list

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.transpose.html b/docs/build/html/python/_autosummary/mlx.core.array.transpose.html new file mode 100644 index 000000000..b8a8ed929 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.transpose.html @@ -0,0 +1,189 @@ + + + + + + + mlx.core.array.transpose — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.transpose

+
+
+array.transpose(self: array, *args, stream: Union[None, Stream, Device] = None) → array
+

Equivalent to transpose() but the axes can be passed either as +a tuple or as separate arguments.

+

See transpose() for full documentation.

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array.var.html b/docs/build/html/python/_autosummary/mlx.core.array.var.html new file mode 100644 index 000000000..1ba8d851a --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array.var.html @@ -0,0 +1,187 @@ + + + + + + + mlx.core.array.var — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array.var

+
+
+array.var(self: array, axis: Union[None, int, List[int]] = None, keepdims: bool = False, ddof: int = 0, *, stream: Union[None, Stream, Device] = None) → array
+

See var().

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.array_equal.html b/docs/build/html/python/_autosummary/mlx.core.array_equal.html new file mode 100644 index 000000000..4a9edbc01 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.array_equal.html @@ -0,0 +1,255 @@ + + + + + + + mlx.core.array_equal — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.array_equal

+
+
+mlx.core.array_equal(a: Union[bool, int, float, complex, array], b: Union[bool, int, float, complex, array], /, *, equal_nan: bool = False, stream: Union[None, Stream, Device] = None) → array
+

Array equality check.

+

Compare two arrays for equality. Returns True if and only if the arrays +have the same shape and their values are equal. The arrays need not have +the same type to be considered equal.

+
+
Parameters:
+
    +

  • a (array) – Input array or scalar.

    • +

  • b (array) – Input array or scalar.

    • +

  • equal_nan (bool) – If True, NaNs are treated as equal. +Defaults to False.

    • +
+
+
Returns:
+

A scalar boolean array.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.broadcast_to.html b/docs/build/html/python/_autosummary/mlx.core.broadcast_to.html new file mode 100644 index 000000000..f796cf4cb --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.broadcast_to.html @@ -0,0 +1,251 @@ + + + + + + + mlx.core.broadcast_to — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.broadcast_to

+
+
+mlx.core.broadcast_to(a: Union[bool, int, float, complex, array], /, shape: List[int], *, stream: Union[None, Stream, Device] = None) → array
+

Broadcast an array to the given shape.

+

The broadcasting semantics are the same as Numpy.

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • shape (list(int)) – The shape to broadcast to.

    • +
+
+
Returns:
+

The output array with the new shape.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.concatenate.html b/docs/build/html/python/_autosummary/mlx.core.concatenate.html new file mode 100644 index 000000000..afd062ce1 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.concatenate.html @@ -0,0 +1,251 @@ + + + + + + + mlx.core.concatenate — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.concatenate

+
+
+mlx.core.concatenate(arrays: List[array], /, axis: Optional[int] = 0, *, stream: Union[None, Stream, Device] = None) → array
+

Concatenate the arrays along the given axis.

+
+
Parameters:
+
    +

  • arrays (list(array)) – Input list or tuple of arrays.

    • +

  • axis (int, optional) – Optional axis to concatenate along. If +unspecified defaults to 0.

    • +
+
+
Returns:
+

The concatenated array.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.conv1d.html b/docs/build/html/python/_autosummary/mlx.core.conv1d.html new file mode 100644 index 000000000..e1f9d7dcb --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.conv1d.html @@ -0,0 +1,255 @@ + + + + + + + mlx.core.conv1d — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.conv1d

+
+
+mlx.core.conv1d(input: array, weight: array, /, stride: int = 1, padding: int = 0, dilation: int = 1, groups: int = 1, *, stream: Union[None, Stream, Device] = None) → array
+

1D convolution over an input with several channels

+

Note: Only the default groups=1 is currently supported.

+
+
Parameters:
+
    +

  • input (array) – input array of shape (N, H, C_in)

    • +

  • weight (array) – weight array of shape (C_out, H, C_in)

    • +

  • stride (int, optional) – kernel stride. Default: 1.

    • +

  • padding (int, optional) – input padding. Default: 0.

    • +

  • dilation (int, optional) – kernel dilation. Default: 1.

    • +

  • groups (int, optional) – input feature groups. Default: 1.

    • +
+
+
Returns:
+

The convolved array.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.conv2d.html b/docs/build/html/python/_autosummary/mlx.core.conv2d.html new file mode 100644 index 000000000..56176e8c7 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.conv2d.html @@ -0,0 +1,261 @@ + + + + + + + mlx.core.conv2d — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.conv2d

+
+
+mlx.core.conv2d(input: array, weight: array, /, stride: Union[int, Tuple[int, int]] = 1, padding: Union[int, Tuple[int, int]] = 0, dilation: Union[int, Tuple[int, int]] = 1, groups: int = 1, *, stream: Union[None, Stream, Device] = None) → array
+

2D convolution over an input with several channels

+

Note: Only the default groups=1 is currently supported.

+
+
Parameters:
+
    +

  • input (array) – input array of shape (N, H, W, C_in)

    • +

  • weight (array) – weight array of shape (C_out, H, W, C_in)

    • +

  • stride (int or tuple(int), optional) – tuple of size 2 with +kernel strides. All spatial dimensions get the same stride if +only one number is specified. Default: 1.

    • +

  • padding (int or tuple(int), optional) – tuple of size 2 with +symmetric input padding. All spatial dimensions get the same +padding if only one number is specified. Default: 0.

    • +

  • dilation (int or tuple(int), optional) – tuple of size 2 with +kernel dilation. All spatial dimensions get the same dilation +if only one number is specified. Default: 1

    • +

  • groups (int, optional) – input feature groups. Default: 1.

    • +
+
+
Returns:
+

The convolved array.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.convolve.html b/docs/build/html/python/_autosummary/mlx.core.convolve.html new file mode 100644 index 000000000..143a9c51c --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.convolve.html @@ -0,0 +1,253 @@ + + + + + + + mlx.core.convolve — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.convolve

+
+
+mlx.core.convolve(a: array, v: array, /, mode: str = 'full', *, stream: Union[None, Stream, Device] = None) → array
+

The discrete convolution of 1D arrays.

+

If v is longer than a, then they are swapped. +The conv filter is flipped following signal processing convention.

+
+
Parameters:
+
    +

  • a (array) – 1D Input array.

    • +

  • v (array) – 1D Input array.

    • +

  • mode (str, optional) – {‘full’, ‘valid’, ‘same’}

    • +
+
+
Returns:
+

The convolved array.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.cos.html b/docs/build/html/python/_autosummary/mlx.core.cos.html new file mode 100644 index 000000000..a200b3dc6 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.cos.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.cos — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.cos

+
+
+mlx.core.cos(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise cosine.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The cosine of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.cosh.html b/docs/build/html/python/_autosummary/mlx.core.cosh.html new file mode 100644 index 000000000..914ccd89b --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.cosh.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.cosh — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.cosh

+
+
+mlx.core.cosh(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise hyperbolic cosine.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The hyperbolic cosine of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.default_device.html b/docs/build/html/python/_autosummary/mlx.core.default_device.html new file mode 100644 index 000000000..f742317e1 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.default_device.html @@ -0,0 +1,158 @@ + + + + + + + mlx.core.default_device — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.default_device

+
+
+mlx.core.default_device() → Device
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.default_stream.html b/docs/build/html/python/_autosummary/mlx.core.default_stream.html new file mode 100644 index 000000000..ff602ae4e --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.default_stream.html @@ -0,0 +1,158 @@ + + + + + + + mlx.core.default_stream — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.default_stream

+
+
+mlx.core.default_stream(device: Device) → Stream
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.divide.html b/docs/build/html/python/_autosummary/mlx.core.divide.html new file mode 100644 index 000000000..b5821d488 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.divide.html @@ -0,0 +1,252 @@ + + + + + + + mlx.core.divide — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.divide

+
+
+mlx.core.divide(a: Union[bool, int, float, complex, array], b: Union[bool, int, float, complex, array], /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise division.

+

Divide two arrays with numpy-style broadcasting semantics. Either or both +input arrays can also be scalars.

+
+
Parameters:
+
    +

  • a (array) – Input array or scalar.

    • +

  • b (array) – Input array or scalar.

    • +
+
+
Returns:
+

The quotient a / b.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.equal.html b/docs/build/html/python/_autosummary/mlx.core.equal.html new file mode 100644 index 000000000..89ca89b48 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.equal.html @@ -0,0 +1,252 @@ + + + + + + + mlx.core.equal — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.equal

+
+
+mlx.core.equal(a: Union[bool, int, float, complex, array], b: Union[bool, int, float, complex, array], /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise equality.

+

Equality comparison on two arrays with numpy-style broadcasting semantics. +Either or both input arrays can also be scalars.

+
+
Parameters:
+
    +

  • a (array) – Input array or scalar.

    • +

  • b (array) – Input array or scalar.

    • +
+
+
Returns:
+

The element-wise comparison a == b.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.erf.html b/docs/build/html/python/_autosummary/mlx.core.erf.html new file mode 100644 index 000000000..40bd777f3 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.erf.html @@ -0,0 +1,250 @@ + + + + + + + mlx.core.erf — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.erf

+
+
+mlx.core.erf(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise error function.

+
+\[\mathrm{erf}(x) = \frac{2}{\sqrt{\pi}} \int_0^t e^{-t^2} \, dx\]
+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The error function of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.erfinv.html b/docs/build/html/python/_autosummary/mlx.core.erfinv.html new file mode 100644 index 000000000..1a3275f3b --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.erfinv.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.erfinv — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.erfinv

+
+
+mlx.core.erfinv(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise inverse of erf().

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The inverse error function of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.eval.html b/docs/build/html/python/_autosummary/mlx.core.eval.html new file mode 100644 index 000000000..28b8c4f95 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.eval.html @@ -0,0 +1,171 @@ + + + + + + + mlx.core.eval — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.eval

+
+
+mlx.core.eval(*args, retain_graph: bool = False) → None
+

Evaluate an array or tree of array.

+
+
Parameters:
+
    +

  • *args (arrays or trees of arrays) – Each argument can be a single array +or a tree of arrays. If a tree is given the nodes can be a Python +list, tuple or dict but the leafs must all be +an array.

    • +

  • retain_graph (bool) – Indicate that the graph structure should be +preserved. This option is intended to enable function transforms +which contain control flow based on the value of an array.

    • +
+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.exp.html b/docs/build/html/python/_autosummary/mlx.core.exp.html new file mode 100644 index 000000000..48720cdcf --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.exp.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.exp — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.exp

+
+
+mlx.core.exp(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise exponential.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The exponential of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.expand_dims.html b/docs/build/html/python/_autosummary/mlx.core.expand_dims.html new file mode 100644 index 000000000..609736561 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.expand_dims.html @@ -0,0 +1,250 @@ + + + + + + + mlx.core.expand_dims — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.expand_dims

+
+
+mlx.core.expand_dims(a: array, /, axis: Union[int, List[int]], *, stream: Union[None, Stream, Device] = None) → array
+

Add a size one dimension at the given axis.

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • axes (int or tuple(int)) – The index of the inserted dimensions.

    • +
+
+
Returns:
+

The array with inserted dimensions.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.fft.fft.html b/docs/build/html/python/_autosummary/mlx.core.fft.fft.html new file mode 100644 index 000000000..09d75c33e --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.fft.fft.html @@ -0,0 +1,182 @@ + + + + + + + mlx.core.fft.fft — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.fft.fft

+
+
+mlx.core.fft.fft(a: array, n: Optional[int] = None, axis: int = - 1, stream: Union[None, Stream, Device] = None) → array
+

One dimensional discrete Fourier Transform.

+
+
Parameters:
+
    +

  • a (array) – The input array.

    • +

  • n (int, optional) – Size of the transformed axis. The +corresponding axis in the input is truncated or padded with +zeros to match n. The default value is a.shape[axis].

    • +

  • axis (int, optional) – Axis along which to perform the FFT. The +default is -1.

    • +
+
+
Returns:
+

The DFT of the input along the given axis.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.fft.fft2.html b/docs/build/html/python/_autosummary/mlx.core.fft.fft2.html new file mode 100644 index 000000000..d27d67740 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.fft.fft2.html @@ -0,0 +1,183 @@ + + + + + + + mlx.core.fft.fft2 — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.fft.fft2

+
+
+mlx.core.fft.fft2(a: array, s: Optional[List[int]] = None, axes: Optional[List[int]] = [- 2, - 1], stream: Union[None, Stream, Device] = None) → array
+

Two dimensional discrete Fourier Transform.

+
+
Parameters:
+
    +

  • a (array) – The input array.

    • +

  • s (list(int), optional) – Sizes of the transformed axes. The +corresponding axes in the input are truncated or padded with +zeros to match the sizes in s. The default value is the +sizes of a along axes.

    • +

  • axes (list(int), optional) – Axes along which to perform the FFT. +The default is [-2, -1].

    • +
+
+
Returns:
+

The DFT of the input along the given axes.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.fft.fftn.html b/docs/build/html/python/_autosummary/mlx.core.fft.fftn.html new file mode 100644 index 000000000..b19d666f6 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.fft.fftn.html @@ -0,0 +1,184 @@ + + + + + + + mlx.core.fft.fftn — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.fft.fftn

+
+
+mlx.core.fft.fftn(a: array, s: Optional[List[int]] = None, axes: Optional[List[int]] = None, stream: Union[None, Stream, Device] = None) → array
+

n-dimensional discrete Fourier Transform.

+
+
Parameters:
+
    +

  • a (array) – The input array.

    • +

  • s (list(int), optional) – Sizes of the transformed axes. The +corresponding axes in the input are truncated or padded with +zeros to match the sizes in s. The default value is the +sizes of a along axes.

    • +

  • axes (list(int), optional) – Axes along which to perform the FFT. +The default is None in which case the FFT is over the last +len(s) axes are or all axes if s is also None.

    • +
+
+
Returns:
+

The DFT of the input along the given axes.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.fft.ifft.html b/docs/build/html/python/_autosummary/mlx.core.fft.ifft.html new file mode 100644 index 000000000..e663fec26 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.fft.ifft.html @@ -0,0 +1,182 @@ + + + + + + + mlx.core.fft.ifft — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.fft.ifft

+
+
+mlx.core.fft.ifft(a: array, n: Optional[int] = None, axis: int = - 1, stream: Union[None, Stream, Device] = None) → array
+

One dimensional inverse discrete Fourier Transform.

+
+
Parameters:
+
    +

  • a (array) – The input array.

    • +

  • n (int, optional) – Size of the transformed axis. The +corresponding axis in the input is truncated or padded with +zeros to match n. The default value is a.shape[axis].

    • +

  • axis (int, optional) – Axis along which to perform the FFT. The +default is -1.

    • +
+
+
Returns:
+

The inverse DFT of the input along the given axis.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.fft.ifft2.html b/docs/build/html/python/_autosummary/mlx.core.fft.ifft2.html new file mode 100644 index 000000000..55fbc414b --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.fft.ifft2.html @@ -0,0 +1,183 @@ + + + + + + + mlx.core.fft.ifft2 — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.fft.ifft2

+
+
+mlx.core.fft.ifft2(a: array, s: Optional[List[int]] = None, axes: Optional[List[int]] = [- 2, - 1], stream: Union[None, Stream, Device] = None) → array
+

Two dimensional inverse discrete Fourier Transform.

+
+
Parameters:
+
    +

  • a (array) – The input array.

    • +

  • s (list(int), optional) – Sizes of the transformed axes. The +corresponding axes in the input are truncated or padded with +zeros to match the sizes in s. The default value is the +sizes of a along axes.

    • +

  • axes (list(int), optional) – Axes along which to perform the FFT. +The default is [-2, -1].

    • +
+
+
Returns:
+

The inverse DFT of the input along the given axes.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.fft.ifftn.html b/docs/build/html/python/_autosummary/mlx.core.fft.ifftn.html new file mode 100644 index 000000000..c507b5f02 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.fft.ifftn.html @@ -0,0 +1,184 @@ + + + + + + + mlx.core.fft.ifftn — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.fft.ifftn

+
+
+mlx.core.fft.ifftn(a: array, s: Optional[List[int]] = None, axes: Optional[List[int]] = None, stream: Union[None, Stream, Device] = None) → array
+

n-dimensional inverse discrete Fourier Transform.

+
+
Parameters:
+
    +

  • a (array) – The input array.

    • +

  • s (list(int), optional) – Sizes of the transformed axes. The +corresponding axes in the input are truncated or padded with +zeros to match the sizes in s. The default value is the +sizes of a along axes.

    • +

  • axes (list(int), optional) – Axes along which to perform the FFT. +The default is None in which case the FFT is over the last +len(s) axes or all axes if s is also None.

    • +
+
+
Returns:
+

The inverse DFT of the input along the given axes.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.fft.irfft.html b/docs/build/html/python/_autosummary/mlx.core.fft.irfft.html new file mode 100644 index 000000000..ac821cc2f --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.fft.irfft.html @@ -0,0 +1,185 @@ + + + + + + + mlx.core.fft.irfft — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.fft.irfft

+
+
+mlx.core.fft.irfft(a: array, n: Optional[int] = None, axis: int = - 1, stream: Union[None, Stream, Device] = None) → array
+

The inverse of rfft().

+

The output has the same shape as the input except along axis in +which case it has size n.

+
+
Parameters:
+
    +

  • a (array) – The input array.

    • +

  • n (int, optional) – Size of the transformed axis. The +corresponding axis in the input is truncated or padded with +zeros to match n // 2 + 1. The default value is +a.shape[axis] // 2 + 1.

    • +

  • axis (int, optional) – Axis along which to perform the FFT. The +default is -1.

    • +
+
+
Returns:
+

The real array containing the inverse of rfft().

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.fft.irfft2.html b/docs/build/html/python/_autosummary/mlx.core.fft.irfft2.html new file mode 100644 index 000000000..e983182e1 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.fft.irfft2.html @@ -0,0 +1,188 @@ + + + + + + + mlx.core.fft.irfft2 — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.fft.irfft2

+
+
+mlx.core.fft.irfft2(a: array, s: Optional[List[int]] = None, axes: Optional[List[int]] = [- 2, - 1], stream: Union[None, Stream, Device] = None) → array
+

The inverse of rfft2().

+

Note the input is generally complex. The dimensions of the input +specified in axes are padded or truncated to match the sizes +from s. The last axis in axes is treated as the real axis +and will have size s[-1] // 2 + 1.

+
+
Parameters:
+
    +

  • a (array) – The input array.

    • +

  • s (list(int), optional) – Sizes of the transformed axes. The +corresponding axes in the input are truncated or padded with +zeros to match the sizes in s except for the last axis +which has size s[-1] // 2 + 1. The default value is the +sizes of a along axes.

    • +

  • axes (list(int), optional) – Axes along which to perform the FFT. +The default is [-2, -1].

    • +
+
+
Returns:
+

The real array containing the inverse of rfft2().

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.fft.irfftn.html b/docs/build/html/python/_autosummary/mlx.core.fft.irfftn.html new file mode 100644 index 000000000..e6db21ded --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.fft.irfftn.html @@ -0,0 +1,188 @@ + + + + + + + mlx.core.fft.irfftn — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.fft.irfftn

+
+
+mlx.core.fft.irfftn(a: array, s: Optional[List[int]] = None, axes: Optional[List[int]] = None, stream: Union[None, Stream, Device] = None) → array
+

The inverse of rfftn().

+

Note the input is generally complex. The dimensions of the input +specified in axes are padded or truncated to match the sizes +from s. The last axis in axes is treated as the real axis +and will have size s[-1] // 2 + 1.

+
+
Parameters:
+
    +

  • a (array) – The input array.

    • +

  • s (list(int), optional) – Sizes of the transformed axes. The +corresponding axes in the input are truncated or padded with +zeros to match the sizes in s. The default value is the +sizes of a along axes.

    • +

  • axes (list(int), optional) – Axes along which to perform the FFT. +The default is None in which case the FFT is over the last +len(s) axes or all axes if s is also None.

    • +
+
+
Returns:
+

The real array containing the inverse of rfftn().

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.fft.rfft.html b/docs/build/html/python/_autosummary/mlx.core.fft.rfft.html new file mode 100644 index 000000000..c3c720fc6 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.fft.rfft.html @@ -0,0 +1,186 @@ + + + + + + + mlx.core.fft.rfft — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.fft.rfft

+
+
+mlx.core.fft.rfft(a: array, n: Optional[int] = None, axis: int = - 1, stream: Union[None, Stream, Device] = None) → array
+

One dimensional discrete Fourier Transform on a real input.

+

The output has the same shape as the input except along axis in +which case it has size n // 2 + 1.

+
+
Parameters:
+
    +

  • a (array) – The input array. If the array is complex it will be silently +cast to a real type.

    • +

  • n (int, optional) – Size of the transformed axis. The +corresponding axis in the input is truncated or padded with +zeros to match n. The default value is a.shape[axis].

    • +

  • axis (int, optional) – Axis along which to perform the FFT. The +default is -1.

    • +
+
+
Returns:
+

The DFT of the input along the given axis. The output +data type will be complex.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.fft.rfft2.html b/docs/build/html/python/_autosummary/mlx.core.fft.rfft2.html new file mode 100644 index 000000000..4219a9e09 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.fft.rfft2.html @@ -0,0 +1,188 @@ + + + + + + + mlx.core.fft.rfft2 — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.fft.rfft2

+
+
+mlx.core.fft.rfft2(a: array, s: Optional[List[int]] = None, axes: Optional[List[int]] = [- 2, - 1], stream: Union[None, Stream, Device] = None) → array
+

Two dimensional real discrete Fourier Transform.

+

The output has the same shape as the input except along the dimensions in +axes in which case it has sizes from s. The last axis in axes is +treated as the real axis and will have size s[-1] // 2 + 1.

+
+
Parameters:
+
    +

  • a (array) – The input array. If the array is complex it will be silently +cast to a real type.

    • +

  • s (list(int), optional) – Sizes of the transformed axes. The +corresponding axes in the input are truncated or padded with +zeros to match the sizes in s. The default value is the +sizes of a along axes.

    • +

  • axes (list(int), optional) – Axes along which to perform the FFT. +The default is [-2, -1].

    • +
+
+
Returns:
+

The real DFT of the input along the given axes. The output +data type will be complex.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.fft.rfftn.html b/docs/build/html/python/_autosummary/mlx.core.fft.rfftn.html new file mode 100644 index 000000000..2ff7459ae --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.fft.rfftn.html @@ -0,0 +1,188 @@ + + + + + + + mlx.core.fft.rfftn — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.fft.rfftn

+
+
+mlx.core.fft.rfftn(a: array, s: Optional[List[int]] = None, axes: Optional[List[int]] = None, stream: Union[None, Stream, Device] = None) → array
+

n-dimensional real discrete Fourier Transform.

+

The output has the same shape as the input except along the dimensions in +axes in which case it has sizes from s. The last axis in axes is +treated as the real axis and will have size s[-1] // 2 + 1.

+
+
Parameters:
+
    +

  • a (array) – The input array. If the array is complex it will be silently +cast to a real type.

    • +

  • s (list(int), optional) – Sizes of the transformed axes. The +corresponding axes in the input are truncated or padded with +zeros to match the sizes in s. The default value is the +sizes of a along axes.

    • +

  • axes (list(int), optional) – Axes along which to perform the FFT. +The default is None in which case the FFT is over the last +len(s) axes or all axes if s is also None.

    • +
+
+
Returns:
+

The real DFT of the input along the given axes. The output

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.full.html b/docs/build/html/python/_autosummary/mlx.core.full.html new file mode 100644 index 000000000..1f1668bff --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.full.html @@ -0,0 +1,254 @@ + + + + + + + mlx.core.full — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.full

+
+
+mlx.core.full(shape: Union[int, List[int]], vals: Union[bool, int, float, complex, array], dtype: Optional[Dtype] = None, *, stream: Union[None, Stream, Device] = None) → array
+

Construct an array with the given value.

+

Constructs an array of size shape filled with vals. If vals +is an array it must be broadcastable to the given shape.

+
+
Parameters:
+
    +

  • shape (int or list(int)) – The shape of the output array.

    • +

  • vals (float or int or array) – Values to fill the array with.

    • +

  • dtype (Dtype, optional) – Data type of the output array. If +unspecified the output type is inferred from vals.

    • +
+
+
Returns:
+

The output array with the specified shape and values.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.grad.html b/docs/build/html/python/_autosummary/mlx.core.grad.html new file mode 100644 index 000000000..21f15fab8 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.grad.html @@ -0,0 +1,177 @@ + + + + + + + mlx.core.grad — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.grad

+
+
+mlx.core.grad(fun: function, argnums: Union[int, List[int]] = 0) → cpp_function
+

Returns a function which computes the gradient of fun.

+
+
Parameters:
+
    +

  • fun (function) – A function which takes a variable number of +array or trees of array and returns +a scalar output array.

    • +

  • argnums (int or list(int), optional) – Specify the index (or indices) +of the arguments of fun to compute the gradient with respect to. If +unspecified this defaults to 0 indicating fun’s first argument.

    • +
+
+
Returns:
+

A function which has the same input arguments as fun and +returns the gradient(s).

+
+
Return type:
+

function

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.greater.html b/docs/build/html/python/_autosummary/mlx.core.greater.html new file mode 100644 index 000000000..6d2476c15 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.greater.html @@ -0,0 +1,252 @@ + + + + + + + mlx.core.greater — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.greater

+
+
+mlx.core.greater(a: Union[bool, int, float, complex, array], b: Union[bool, int, float, complex, array], /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise greater than.

+

Strict greater than on two arrays with numpy-style broadcasting semantics. +Either or both input arrays can also be scalars.

+
+
Parameters:
+
    +

  • a (array) – Input array or scalar.

    • +

  • b (array) – Input array or scalar.

    • +
+
+
Returns:
+

The element-wise comparison a > b.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.greater_equal.html b/docs/build/html/python/_autosummary/mlx.core.greater_equal.html new file mode 100644 index 000000000..06defed9c --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.greater_equal.html @@ -0,0 +1,252 @@ + + + + + + + mlx.core.greater_equal — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.greater_equal

+
+
+mlx.core.greater_equal(a: Union[bool, int, float, complex, array], b: Union[bool, int, float, complex, array], /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise greater or equal.

+

Greater than or equal on two arrays with numpy-style broadcasting semantics. +Either or both input arrays can also be scalars.

+
+
Parameters:
+
    +

  • a (array) – Input array or scalar.

    • +

  • b (array) – Input array or scalar.

    • +
+
+
Returns:
+

The element-wise comparison a >= b.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.jvp.html b/docs/build/html/python/_autosummary/mlx.core.jvp.html new file mode 100644 index 000000000..aba1e61a8 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.jvp.html @@ -0,0 +1,180 @@ + + + + + + + mlx.core.jvp — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.jvp

+
+
+mlx.core.jvp(fun: function, primals: List[array], tangents: List[array]) → Tuple[List[array], List[array]]
+

Compute the Jacobian-vector product.

+

This computes the product of the Jacobian of a function fun evaluated +at primals with the tangents.

+
+
Parameters:
+
    +

  • fun (function) – A function which takes a variable number of array +and returns a single array or list of array.

    • +

  • primals (list(array)) – A list of array at which to +evaluate the Jacobian.

    • +

  • tangents (list(array)) – A list of array which are the +“vector” in the Jacobian-vector product. The tangents should be the +same in number, shape, and type as the inputs of fun (i.e. the primals).

    • +
+
+
Returns:
+

A list of the Jacobian-vector products which +is the same in number, shape, and type of the inputs to fun.

+
+
Return type:
+

list(array)

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.less.html b/docs/build/html/python/_autosummary/mlx.core.less.html new file mode 100644 index 000000000..2f76dd5b9 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.less.html @@ -0,0 +1,252 @@ + + + + + + + mlx.core.less — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.less

+
+
+mlx.core.less(a: Union[bool, int, float, complex, array], b: Union[bool, int, float, complex, array], /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise less than.

+

Strict less than on two arrays with numpy-style broadcasting semantics. +Either or both input arrays can also be scalars.

+
+
Parameters:
+
    +

  • a (array) – Input array or scalar.

    • +

  • b (array) – Input array or scalar.

    • +
+
+
Returns:
+

The element-wise comparison a < b.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.less_equal.html b/docs/build/html/python/_autosummary/mlx.core.less_equal.html new file mode 100644 index 000000000..a12588022 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.less_equal.html @@ -0,0 +1,252 @@ + + + + + + + mlx.core.less_equal — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.less_equal

+
+
+mlx.core.less_equal(a: Union[bool, int, float, complex, array], b: Union[bool, int, float, complex, array], /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise less than or equal.

+

Less than or equal on two arrays with numpy-style broadcasting semantics. +Either or both input arrays can also be scalars.

+
+
Parameters:
+
    +

  • a (array) – Input array or scalar.

    • +

  • b (array) – Input array or scalar.

    • +
+
+
Returns:
+

The element-wise comparison a <= b.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.load.html b/docs/build/html/python/_autosummary/mlx.core.load.html new file mode 100644 index 000000000..c94f3936d --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.load.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.load — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.load

+
+
+mlx.core.load(file: object, /, *, stream: Union[None, Stream, Device] = None) → Union[array, Dict[str, array]]
+

Load array(s) from a binary file in .npy or .npz format.

+
+
Parameters:
+

file (file, str) – File in which the array is saved

+
+
Returns:
+

The loaded array if .npy file or a dict mapping name to array if .npz file

+
+
Return type:
+

result (array, dict)

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.log.html b/docs/build/html/python/_autosummary/mlx.core.log.html new file mode 100644 index 000000000..efe99a808 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.log.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.log — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.log

+
+
+mlx.core.log(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise natural logarithm.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The natural logarithm of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.log10.html b/docs/build/html/python/_autosummary/mlx.core.log10.html new file mode 100644 index 000000000..69b180646 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.log10.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.log10 — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.log10

+
+
+mlx.core.log10(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise base-10 logarithm.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The base-10 logarithm of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.log1p.html b/docs/build/html/python/_autosummary/mlx.core.log1p.html new file mode 100644 index 000000000..6d3b39bd1 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.log1p.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.log1p — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.log1p

+
+
+mlx.core.log1p(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise natural log of one plus the array.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The natural logarithm of one plus a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.log2.html b/docs/build/html/python/_autosummary/mlx.core.log2.html new file mode 100644 index 000000000..5e32d04fd --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.log2.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.log2 — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.log2

+
+
+mlx.core.log2(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise base-2 logarithm.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The base-2 logarithm of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.logaddexp.html b/docs/build/html/python/_autosummary/mlx.core.logaddexp.html new file mode 100644 index 000000000..985aaeb46 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.logaddexp.html @@ -0,0 +1,253 @@ + + + + + + + mlx.core.logaddexp — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.logaddexp

+
+
+mlx.core.logaddexp(a: Union[bool, int, float, complex, array], b: Union[bool, int, float, complex, array], /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise log-add-exp.

+

This is a numerically stable log-add-exp of two arrays with numpy-style +broadcasting semantics. Either or both input arrays can also be scalars.

+

The computation is is a numerically stable version of log(exp(a) + exp(b)).

+
+
Parameters:
+
    +

  • a (array) – Input array or scalar.

    • +

  • b (array) – Input array or scalar.

    • +
+
+
Returns:
+

The log-add-exp of a and b.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.logical_not.html b/docs/build/html/python/_autosummary/mlx.core.logical_not.html new file mode 100644 index 000000000..954f4aef6 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.logical_not.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.logical_not — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.logical_not

+
+
+mlx.core.logical_not(a: Union[bool, int, float, complex, array], /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise logical not.

+
+
Parameters:
+

a (array) – Input array or scalar.

+
+
Returns:
+

The boolean array containing the logical not of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.logsumexp.html b/docs/build/html/python/_autosummary/mlx.core.logsumexp.html new file mode 100644 index 000000000..f37b9ce9d --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.logsumexp.html @@ -0,0 +1,258 @@ + + + + + + + mlx.core.logsumexp — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.logsumexp

+
+
+mlx.core.logsumexp(a: array, /, axis: Union[None, int, List[int]] = None, keepdims: bool = False, *, stream: Union[None, Stream, Device] = None) → array
+

A log-sum-exp reduction over the given axes.

+

The log-sum-exp reduction is a numerically stable version of:

+
log(sum(exp(a), axis))
+
+
+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • axis (int or list(int), optional) – Optional axis or +axes to reduce over. If unspecified this defaults +to reducing over the entire array.

    • +

  • keepdims (bool, optional) – Keep reduced axes as +singleton dimensions, defaults to False.

    • +
+
+
Returns:
+

The output array with the corresponding axes reduced.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.matmul.html b/docs/build/html/python/_autosummary/mlx.core.matmul.html new file mode 100644 index 000000000..22be52e8a --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.matmul.html @@ -0,0 +1,263 @@ + + + + + + + mlx.core.matmul — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.matmul

+
+
+mlx.core.matmul(a: array, b: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Matrix multiplication.

+

Perform the (possibly batched) matrix multiplication of two arrays. This function supports +broadcasting for arrays with more than two dimensions.

+
    +

  • If the first array is 1-D then a 1 is prepended to its shape to make it +a matrix. Similarly if the second array is 1-D then a 1 is appended to its +shape to make it a matrix. In either case the singleton dimension is removed +from the result.

    • +

  • A batched matrix multiplication is performed if the arrays have more than +2 dimensions. The matrix dimensions for the matrix product are the last +two dimensions of each input.

    • +

  • All but the last two dimensions of each input are broadcast with one another using +standard numpy-style broadcasting semantics.

    • +
+
+
Parameters:
+
    +

  • a (array) – Input array or scalar.

    • +

  • b (array) – Input array or scalar.

    • +
+
+
Returns:
+

The matrix product of a and b.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.max.html b/docs/build/html/python/_autosummary/mlx.core.max.html new file mode 100644 index 000000000..755347280 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.max.html @@ -0,0 +1,254 @@ + + + + + + + mlx.core.max — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.max

+
+
+mlx.core.max(a: array, /, axis: Union[None, int, List[int]] = None, keepdims: bool = False, *, stream: Union[None, Stream, Device] = None) → array
+

An max reduction over the given axes.

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • axis (int or list(int), optional) – Optional axis or +axes to reduce over. If unspecified this defaults +to reducing over the entire array.

    • +

  • keepdims (bool, optional) – Keep reduced axes as +singleton dimensions, defaults to False.

    • +
+
+
Returns:
+

The output array with the corresponding axes reduced.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.maximum.html b/docs/build/html/python/_autosummary/mlx.core.maximum.html new file mode 100644 index 000000000..c009566bf --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.maximum.html @@ -0,0 +1,252 @@ + + + + + + + mlx.core.maximum — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.maximum

+
+
+mlx.core.maximum(a: Union[bool, int, float, complex, array], b: Union[bool, int, float, complex, array], /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise maximum.

+

Take the element-wise max of two arrays with numpy-style broadcasting +semantics. Either or both input arrays can also be scalars.

+
+
Parameters:
+
    +

  • a (array) – Input array or scalar.

    • +

  • b (array) – Input array or scalar.

    • +
+
+
Returns:
+

The max of a and b.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.mean.html b/docs/build/html/python/_autosummary/mlx.core.mean.html new file mode 100644 index 000000000..a6726ae66 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.mean.html @@ -0,0 +1,254 @@ + + + + + + + mlx.core.mean — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.mean

+
+
+mlx.core.mean(a: array, /, axis: Union[None, int, List[int]] = None, keepdims: bool = False, *, stream: Union[None, Stream, Device] = None) → array
+

Compute the mean(s) over the given axes.

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • axis (int or list(int), optional) – Optional axis or +axes to reduce over. If unspecified this defaults +to reducing over the entire array.

    • +

  • keepdims (bool, optional) – Keep reduced axes as +singleton dimensions, defaults to False.

    • +
+
+
Returns:
+

The output array of means.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.min.html b/docs/build/html/python/_autosummary/mlx.core.min.html new file mode 100644 index 000000000..b73058bf8 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.min.html @@ -0,0 +1,254 @@ + + + + + + + mlx.core.min — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.min

+
+
+mlx.core.min(a: array, /, axis: Union[None, int, List[int]] = None, keepdims: bool = False, *, stream: Union[None, Stream, Device] = None) → array
+

An min reduction over the given axes.

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • axis (int or list(int), optional) – Optional axis or +axes to reduce over. If unspecified this defaults +to reducing over the entire array.

    • +

  • keepdims (bool, optional) – Keep reduced axes as +singleton dimensions, defaults to False.

    • +
+
+
Returns:
+

The output array with the corresponding axes reduced.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.minimum.html b/docs/build/html/python/_autosummary/mlx.core.minimum.html new file mode 100644 index 000000000..a41ce6387 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.minimum.html @@ -0,0 +1,252 @@ + + + + + + + mlx.core.minimum — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.minimum

+
+
+mlx.core.minimum(a: Union[bool, int, float, complex, array], b: Union[bool, int, float, complex, array], /, stream: Union[None, Stream, Device] = None) → array
+

Element-wise minimum.

+

Take the element-wise min of two arrays with numpy-style broadcasting +semantics. Either or both input arrays can also be scalars.

+
+
Parameters:
+
    +

  • a (array) – Input array or scalar.

    • +

  • b (array) – Input array or scalar.

    • +
+
+
Returns:
+

The min of a and b.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.multiply.html b/docs/build/html/python/_autosummary/mlx.core.multiply.html new file mode 100644 index 000000000..5e55ccf40 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.multiply.html @@ -0,0 +1,252 @@ + + + + + + + mlx.core.multiply — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.multiply

+
+
+mlx.core.multiply(a: Union[bool, int, float, complex, array], b: Union[bool, int, float, complex, array], /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise multiplication.

+

Multiply two arrays with numpy-style broadcasting semantics. Either or both +input arrays can also be scalars.

+
+
Parameters:
+
    +

  • a (array) – Input array or scalar.

    • +

  • b (array) – Input array or scalar.

    • +
+
+
Returns:
+

The multiplication a * b.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.negative.html b/docs/build/html/python/_autosummary/mlx.core.negative.html new file mode 100644 index 000000000..af83aeeb7 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.negative.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.negative — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.negative

+
+
+mlx.core.negative(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise negation.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The negative of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.new_stream.html b/docs/build/html/python/_autosummary/mlx.core.new_stream.html new file mode 100644 index 000000000..dcfcbcc4f --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.new_stream.html @@ -0,0 +1,158 @@ + + + + + + + mlx.core.new_stream — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.new_stream

+
+
+mlx.core.new_stream(device: Device) → Stream
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.ones.html b/docs/build/html/python/_autosummary/mlx.core.ones.html new file mode 100644 index 000000000..1cb41f5ef --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.ones.html @@ -0,0 +1,251 @@ + + + + + + + mlx.core.ones — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.ones

+
+
+mlx.core.ones(shape: Union[int, List[int]], dtype: Optional[Dtype] = None, *, stream: Union[None, Stream, Device] = None) → array
+

Construct an array of ones.

+
+
Parameters:
+
    +

  • shape (int or list(int)) – The shape of the output array.

    • +

  • dtype (Dtype, optional) – Data type of the output array. If +unspecified the output type defaults to float32.

    • +
+
+
Returns:
+

The array of ones with the specified shape.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.ones_like.html b/docs/build/html/python/_autosummary/mlx.core.ones_like.html new file mode 100644 index 000000000..98506ee92 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.ones_like.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.ones_like — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.ones_like

+
+
+mlx.core.ones_like(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

An array of ones like the input.

+
+
Parameters:
+

a (array) – The input to take the shape and type from.

+
+
Returns:
+

The output array filled with ones.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.pad.html b/docs/build/html/python/_autosummary/mlx.core.pad.html new file mode 100644 index 000000000..660f09136 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.pad.html @@ -0,0 +1,257 @@ + + + + + + + mlx.core.pad — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.pad

+
+
+mlx.core.pad(a: array, /, pad_width: Union[int, Tuple[int], Tuple[int, int], List[Tuple[int, int]]], constant_values: Union[bool, int, float, complex, array] = 0, *, stream: Union[None, Stream, Device] = None) → array
+

Pad an array with a constant value

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • pad_width (int, tuple(int), tuple(int, int) or list(tuple(int, int))) – Number of padded +values to add to the edges of each axis:((before_1, after_1), +(before_2, after_2), ..., (before_N, after_N)). If a single pair +of integers is passed then (before_i, after_i) are all the same. +If a single integer or tuple with a single integer is passed then +all axes are extended by the same number on each side.

    • +

  • constant_value (array or scalar, optional) – Optional constant value +to pad the edges of the array with.

    • +
+
+
Returns:
+

The padded array.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.partition.html b/docs/build/html/python/_autosummary/mlx.core.partition.html new file mode 100644 index 000000000..a3f450410 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.partition.html @@ -0,0 +1,258 @@ + + + + + + + mlx.core.partition — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.partition

+
+
+mlx.core.partition(a: array, /, kth: int, axis: Optional[int] = - 1, *, stream: Union[None, Stream, Device] = None) → array
+

Returns a partitioned copy of the array such that the smaller kth +elements are first.

+

The ordering of the elements in partitions is undefined.

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • kth (int) – Element at the kth index will be in its sorted +position in the output. All elements before the kth index will +be less or equal to the kth element and all elements after +will be greater or equal to the kth element in the output.

    • +

  • axis (int or None, optional) – Optional axis to partition over. +If None, this partitions over the flattened array. +If unspecified, it defaults to -1.

    • +
+
+
Returns:
+

The partitioned array.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.prod.html b/docs/build/html/python/_autosummary/mlx.core.prod.html new file mode 100644 index 000000000..40e6d7399 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.prod.html @@ -0,0 +1,254 @@ + + + + + + + mlx.core.prod — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.prod

+
+
+mlx.core.prod(a: array, /, axis: Union[None, int, List[int]] = None, keepdims: bool = False, *, stream: Union[None, Stream, Device] = None) → array
+

An product reduction over the given axes.

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • axis (int or list(int), optional) – Optional axis or +axes to reduce over. If unspecified this defaults +to reducing over the entire array.

    • +

  • keepdims (bool, optional) – Keep reduced axes as +singleton dimensions, defaults to False.

    • +
+
+
Returns:
+

The output array with the corresponding axes reduced.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.random.bernoulli.html b/docs/build/html/python/_autosummary/mlx.core.random.bernoulli.html new file mode 100644 index 000000000..ce6405eca --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.random.bernoulli.html @@ -0,0 +1,182 @@ + + + + + + + mlx.core.random.bernoulli — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.random.bernoulli

+
+
+mlx.core.random.bernoulli(p: Union[bool, int, float, complex, array] = 0.5, shape: Optional[List[int]] = None, key: Optional[array] = None, stream: Union[None, Stream, Device] = None) → array
+

Generate Bernoulli random values.

+

The values are sampled from the bernoulli distribution with parameter +p. The parameter p can be a float or array and +must be broadcastable to shape.

+
+
Parameters:
+
    +

  • p (float or array, optional) – Parameter of the Bernoulli +distribution. Default is 0.5.

    • +

  • shape (list(int), optional) – Shape of the output. The default +shape is p.shape.

    • +

  • key (array, optional) – A PRNG key. Default: None.

    • +
+
+
Returns:
+

The array of random integers.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.random.categorical.html b/docs/build/html/python/_autosummary/mlx.core.random.categorical.html new file mode 100644 index 000000000..142939bd2 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.random.categorical.html @@ -0,0 +1,188 @@ + + + + + + + mlx.core.random.categorical — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.random.categorical

+
+
+mlx.core.random.categorical(logits: array, axis: int = - 1, shape: Optional[List[int]] = None, num_samples: Optional[int] = None, key: Optional[array] = None, stream: Union[None, Stream, Device] = None) → array
+

Sample from a categorical distribution.

+

The values are sampled from the categorical distribution specified by +the unnormalized values in logits. Note, at most one of shape +or num_samples can be specified. If both are None, the output +has the same shape as logits with the axis dimension removed.

+
+
Parameters:
+
    +

  • logits (array) – The unnormalized categorical distribution(s).

    • +

  • axis (int, optional) – The axis which specifies the distribution. +Default is -1.

    • +

  • shape (list(int), optional) – The shape of the output. This must +be broadcast compatable with logits.shape with the axis +dimension removed. Default: None

    • +

  • num_samples (int, optional) – The number of samples to draw from each +of the categorical distributions in logits. The output will have +num_samples in the last dimension. Default: None.

    • +

  • key (array, optional) – A PRNG key. Default: None.

    • +
+
+
Returns:
+

The shape-sized output array with type uint32.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.random.gumbel.html b/docs/build/html/python/_autosummary/mlx.core.random.gumbel.html new file mode 100644 index 000000000..8be03ed8b --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.random.gumbel.html @@ -0,0 +1,182 @@ + + + + + + + mlx.core.random.gumbel — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.random.gumbel

+
+
+mlx.core.random.gumbel(shape: List[int] = [], dtype: Dtype = float32, stream: Optional[array] = None, key: Union[None, Stream, Device] = None) → array
+

Sample from the standard Gumbel distribution.

+

The values are sampled from a standard Gumbel distribution +which CDF exp(-exp(-x)).

+
+
Parameters:
+
    +

  • shape (list(int)) – The shape of the output.

    • +

  • key (array, optional) – A PRNG key. Default: None.

    • +
+
+
Returns:
+

+
The array with shape shape and

distributed according to the Gumbel distribution

+
+
+

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.random.key.html b/docs/build/html/python/_autosummary/mlx.core.random.key.html new file mode 100644 index 000000000..44e35b990 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.random.key.html @@ -0,0 +1,173 @@ + + + + + + + mlx.core.random.key — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.random.key

+
+
+mlx.core.random.key(seed: int) → array
+

Get a PRNG key from a seed.

+
+
Parameters:
+

seed (int) – Seed for the PRNG.

+
+
Returns:
+

The PRNG key array.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.random.normal.html b/docs/build/html/python/_autosummary/mlx.core.random.normal.html new file mode 100644 index 000000000..5c0658149 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.random.normal.html @@ -0,0 +1,177 @@ + + + + + + + mlx.core.random.normal — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.random.normal

+
+
+mlx.core.random.normal(shape: List[int] = [], dtype: Dtype = float32, key: Optional[array] = None, stream: Union[None, Stream, Device] = None) → array
+

Generate normally distributed random numbers.

+
+
Parameters:
+
    +

  • shape (list(int), optional) – Shape of the output. Default is ().

    • +

  • dtype (Dtype, optional) – Type of the output. Default is float32.

    • +

  • key (array, optional) – A PRNG key. Default: None.

    • +
+
+
Returns:
+

The output array of random values.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.random.randint.html b/docs/build/html/python/_autosummary/mlx.core.random.randint.html new file mode 100644 index 000000000..42a26c70c --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.random.randint.html @@ -0,0 +1,182 @@ + + + + + + + mlx.core.random.randint — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.random.randint

+
+
+mlx.core.random.randint(low: Union[bool, int, float, complex, array], high: Union[bool, int, float, complex, array], shape: List[int] = [], dtype: Dtype = int32, key: Optional[array] = None, stream: Union[None, Stream, Device] = None) → array
+

Generate random integers from the given interval.

+

The values are sampled with equal probability from the integers in +half-open interval [low, high). The lower and upper bound can be +scalars or arrays and must be roadcastable to shape.

+
+
Parameters:
+
    +

  • low (scalar or array) – Lower bound of the interval.

    • +

  • high (scalar or array) – Upper bound of the interval.

    • +

  • shape (list(int), optional) – Shape of the output. Defaults to ().

    • +

  • dtype (Dtype, optional) – Type of the output. Defaults to int32.

    • +

  • key (array, optional) – A PRNG key. Default: None.

    • +
+
+
Returns:
+

The array of random integers.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.random.seed.html b/docs/build/html/python/_autosummary/mlx.core.random.seed.html new file mode 100644 index 000000000..c7d96c3d8 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.random.seed.html @@ -0,0 +1,167 @@ + + + + + + + mlx.core.random.seed — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.random.seed

+
+
+mlx.core.random.seed(seed: int) → None
+

Seed the global PRNG.

+
+
Parameters:
+

seed (int) – Seed for the global PRNG.

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.random.split.html b/docs/build/html/python/_autosummary/mlx.core.random.split.html new file mode 100644 index 000000000..21153a76d --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.random.split.html @@ -0,0 +1,176 @@ + + + + + + + mlx.core.random.split — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.random.split

+
+
+mlx.core.random.split(key: array, num: int = 2, stream: Union[None, Stream, Device] = None) → array
+

Split a PRNG key into sub keys.

+
+
Parameters:
+
    +

  • key (array) – Input key to split.

    • +

  • num (int, optional) – Number of sub keys. Default is 2.

    • +
+
+
Returns:
+

The array of sub keys with num as its first dimension.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.random.truncated_normal.html b/docs/build/html/python/_autosummary/mlx.core.random.truncated_normal.html new file mode 100644 index 000000000..1a68309f9 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.random.truncated_normal.html @@ -0,0 +1,184 @@ + + + + + + + mlx.core.random.truncated_normal — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.random.truncated_normal

+
+
+mlx.core.random.truncated_normal(lower: Union[bool, int, float, complex, array], upper: Union[bool, int, float, complex, array], shape: Optional[List[int]] = None, dtype: Dtype = float32, key: Optional[array] = None, stream: Union[None, Stream, Device] = None) → array
+

Generate values from a truncated normal distribution.

+

The values are sampled from the truncated normal distribution +on the domain (lower, upper). The bounds lower and upper +can be scalars or arrays and must be broadcastable to shape.

+
+
Parameters:
+
    +

  • lower (scalar or array) – Lower bound of the domain.

    • +

  • upper (scalar or array) – Upper bound of the domain.

    • +

  • shape (list(int), optional) – The shape of the output. +Default is ().

    • +

  • dtype (Dtype, optinoal) – The data type of the output. +Default is float32.

    • +

  • key (array, optional) – A PRNG key. Default: None.

    • +
+
+
Returns:
+

The output array of random values.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.random.uniform.html b/docs/build/html/python/_autosummary/mlx.core.random.uniform.html new file mode 100644 index 000000000..1d5ca285c --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.random.uniform.html @@ -0,0 +1,182 @@ + + + + + + + mlx.core.random.uniform — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.random.uniform

+
+
+mlx.core.random.uniform(low: Union[bool, int, float, complex, array] = 0, high: Union[bool, int, float, complex, array] = 1, shape: List[int] = [], dtype: Dtype = float32, key: Optional[array] = None, stream: Union[None, Stream, Device] = None) → array
+

Generate uniformly distributed random numbers.

+

The values are sampled uniformly in the half-open interval [low, high). +The lower and upper bound can be scalars or arrays and must be +broadcastable to shape.

+
+
Parameters:
+
    +

  • low (scalar or array, optional) – Lower bound of the distribution. Default is 0.

    • +

  • high (scalar or array, optional) – Upper bound of the distribution. Default is 1.

    • +

  • shape (list(int), optional) – Shape of the output. Default is ().

    • +

  • key (array, optional) – A PRNG key. Default: None.

    • +

  • dtype (Dtype, optional) – Type of the output. Default is float32.

    • +
+
+
Returns:
+

The output array random values.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.reciprocal.html b/docs/build/html/python/_autosummary/mlx.core.reciprocal.html new file mode 100644 index 000000000..3fc85e319 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.reciprocal.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.reciprocal — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.reciprocal

+
+
+mlx.core.reciprocal(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise reciprocal.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The reciprocal of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.reshape.html b/docs/build/html/python/_autosummary/mlx.core.reshape.html new file mode 100644 index 000000000..51a0391e2 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.reshape.html @@ -0,0 +1,252 @@ + + + + + + + mlx.core.reshape — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.reshape

+
+
+mlx.core.reshape(a: array, /, shape: List[int], *, stream: Union[None, Stream, Device] = None) → array
+

Reshape an array while preserving the size.

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • shape (tuple(int)) – New shape.

    • +

  • stream (Stream, optional) – Stream or device. Defaults to `None` +in which case the default stream of the default device is used.

    • +
+
+
Returns:
+

The reshaped array.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.rsqrt.html b/docs/build/html/python/_autosummary/mlx.core.rsqrt.html new file mode 100644 index 000000000..423f0bc0f --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.rsqrt.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.rsqrt — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.rsqrt

+
+
+mlx.core.rsqrt(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise reciprocal and square root.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

One over the square root of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.save.html b/docs/build/html/python/_autosummary/mlx.core.save.html new file mode 100644 index 000000000..849dd3de5 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.save.html @@ -0,0 +1,246 @@ + + + + + + + mlx.core.save — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.save

+
+
+mlx.core.save(file: object, arr: array, /, retain_graph: bool = True) → None
+

Save the array to a binary file in .npy format.

+
+
Parameters:
+
    +

  • file (str) – File to which the array is saved

    • +

  • arr (array) – Array to be saved.

    • +

  • retain_graph (bool) – Optional argument to retain graph +during array evaluation before saving. Default: True

    • +
+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.savez.html b/docs/build/html/python/_autosummary/mlx.core.savez.html new file mode 100644 index 000000000..84d103bf5 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.savez.html @@ -0,0 +1,259 @@ + + + + + + + mlx.core.savez — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.savez

+
+
+mlx.core.savez(file: object, /, *args, **kwargs) → None
+

Save several arrays to a binary file in uncompressed .npz format.

+
import mlx.core as mx
+
+x = mx.ones((10, 10))
+mx.savez("my_path.npz", x=x)
+
+import mlx.nn as nn
+from mlx.utils import tree_flatten
+
+model = nn.TransformerEncoder(6, 128, 4)
+flat_params = tree_flatten(model.parameters())
+mx.savez("model.npz", **dict(flat_params))
+
+
+
+
Parameters:
+
    +

  • file (file, str) – Path to file to which the arrays are saved.

    • +

  • args (arrays) – Arrays to be saved.

    • +

  • kwargs (arrays) – Arrays to be saved. Each array will be saved +with the associated keyword as the output file name.

    • +
+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.savez_compressed.html b/docs/build/html/python/_autosummary/mlx.core.savez_compressed.html new file mode 100644 index 000000000..b219bd163 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.savez_compressed.html @@ -0,0 +1,246 @@ + + + + + + + mlx.core.savez_compressed — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.savez_compressed

+
+
+mlx.core.savez_compressed(file: object, /, *args, **kwargs) → None
+

Save several arrays to a binary file in compressed .npz format.

+
+
Parameters:
+
    +

  • file (file, str) – Path to file to which the arrays are saved.

    • +

  • args (arrays) – Arrays to be saved.

    • +

  • kwargs (arrays) – Arrays to be saved. Each array will be saved +with the associated keyword as the output file name.

    • +
+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.set_default_device.html b/docs/build/html/python/_autosummary/mlx.core.set_default_device.html new file mode 100644 index 000000000..52de33f86 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.set_default_device.html @@ -0,0 +1,158 @@ + + + + + + + mlx.core.set_default_device — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.set_default_device

+
+
+mlx.core.set_default_device(device: Device) → None
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.set_default_stream.html b/docs/build/html/python/_autosummary/mlx.core.set_default_stream.html new file mode 100644 index 000000000..c250c00a7 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.set_default_stream.html @@ -0,0 +1,158 @@ + + + + + + + mlx.core.set_default_stream — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.set_default_stream

+
+
+mlx.core.set_default_stream(stream: Stream) → None
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.sigmoid.html b/docs/build/html/python/_autosummary/mlx.core.sigmoid.html new file mode 100644 index 000000000..d47baa3e8 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.sigmoid.html @@ -0,0 +1,251 @@ + + + + + + + mlx.core.sigmoid — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.sigmoid

+
+
+mlx.core.sigmoid(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise logistic sigmoid.

+

The logistic sigmoid function is:

+
+\[\mathrm{sigmoid}(x) = \frac{1}{1 + e^{-x}}\]
+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The logistic sigmoid of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.sign.html b/docs/build/html/python/_autosummary/mlx.core.sign.html new file mode 100644 index 000000000..2bcde2986 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.sign.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.sign — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.sign

+
+
+mlx.core.sign(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise sign.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The sign of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.sin.html b/docs/build/html/python/_autosummary/mlx.core.sin.html new file mode 100644 index 000000000..26b5a67c7 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.sin.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.sin — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.sin

+
+
+mlx.core.sin(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise sine.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The sine of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.sinh.html b/docs/build/html/python/_autosummary/mlx.core.sinh.html new file mode 100644 index 000000000..113334aeb --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.sinh.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.sinh — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.sinh

+
+
+mlx.core.sinh(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise hyperbolic sine.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The hyperbolic sine of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.softmax.html b/docs/build/html/python/_autosummary/mlx.core.softmax.html new file mode 100644 index 000000000..f03fd789c --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.softmax.html @@ -0,0 +1,256 @@ + + + + + + + mlx.core.softmax — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.softmax

+
+
+mlx.core.softmax(a: array, /, axis: Union[None, int, List[int]] = None, *, stream: Union[None, Stream, Device] = None) → array
+

Perform the softmax along the given axis.

+

This operation is a numerically stable version of:

+
exp(a) / sum(exp(a), axis, keepdims=True)
+
+
+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • axis (int or list(int), optional) – Optional axis or axes to compute +the softmax over. If unspecified this performs the softmax over +the full array.

    • +
+
+
Returns:
+

The output of the softmax.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.sort.html b/docs/build/html/python/_autosummary/mlx.core.sort.html new file mode 100644 index 000000000..1854bdf06 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.sort.html @@ -0,0 +1,252 @@ + + + + + + + mlx.core.sort — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.sort

+
+
+mlx.core.sort(a: array, /, axis: Optional[int] = - 1, *, stream: Union[None, Stream, Device] = None) → array
+

Returns a sorted copy of the array.

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • axis (int or None, optional) – Optional axis to sort over. +If None, this sorts over the flattened array. +If unspecified, it defaults to -1 (sorting over the last axis).

    • +
+
+
Returns:
+

The sorted array.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.split.html b/docs/build/html/python/_autosummary/mlx.core.split.html new file mode 100644 index 000000000..7397b2c29 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.split.html @@ -0,0 +1,255 @@ + + + + + + + mlx.core.split — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.split

+
+
+mlx.core.split(a: array, /, indices_or_sections: Union[int, List[int]], axis: int = 0, *, stream: Union[None, Stream, Device] = None) → List[array]
+

Split an array along a given axis.

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • indices_or_sections (int or list(int)) – If indices_or_sections +is an integer the array is split into that many sections of equal +size. An error is raised if this is not possible. If indices_or_sections +is a list, the list contains the indices of the start of each subarray +along the given axis.

    • +

  • axis (int, optional) – Axis to split along, defaults to 0.

    • +
+
+
Returns:
+

A list of split arrays.

+
+
Return type:
+

list(array)

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.sqrt.html b/docs/build/html/python/_autosummary/mlx.core.sqrt.html new file mode 100644 index 000000000..66f137d56 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.sqrt.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.sqrt — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.sqrt

+
+
+mlx.core.sqrt(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise square root.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The square root of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.square.html b/docs/build/html/python/_autosummary/mlx.core.square.html new file mode 100644 index 000000000..fca7636ae --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.square.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.square — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.square

+
+
+mlx.core.square(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise square.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The square of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.squeeze.html b/docs/build/html/python/_autosummary/mlx.core.squeeze.html new file mode 100644 index 000000000..9bafd2b64 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.squeeze.html @@ -0,0 +1,251 @@ + + + + + + + mlx.core.squeeze — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.squeeze

+
+
+mlx.core.squeeze(a: array, /, axis: Union[None, int, List[int]] = None, *, stream: Union[None, Stream, Device] = None) → array
+

Remove length one axes from an array.

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • axis (int or tuple(int), optional) – Axes to remove. Defaults

    • +

  • removed. (to `None` in which case all size one axes are) –

    • +
+
+
Returns:
+

The output array with size one axes removed.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.stop_gradient.html b/docs/build/html/python/_autosummary/mlx.core.stop_gradient.html new file mode 100644 index 000000000..0227398ef --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.stop_gradient.html @@ -0,0 +1,253 @@ + + + + + + + mlx.core.stop_gradient — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.stop_gradient

+
+
+mlx.core.stop_gradient(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Stop gradients from being computed.

+

The operation is the identity but it prevents gradients from flowing +through the array.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

+
The unchanged input a but without gradient flowing

through it.

+
+
+

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.subtract.html b/docs/build/html/python/_autosummary/mlx.core.subtract.html new file mode 100644 index 000000000..893bc8ccc --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.subtract.html @@ -0,0 +1,252 @@ + + + + + + + mlx.core.subtract — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.subtract

+
+
+mlx.core.subtract(a: Union[bool, int, float, complex, array], b: Union[bool, int, float, complex, array], /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise subtraction.

+

Subtract one array from another with numpy-style broadcasting semantics. Either or both +input arrays can also be scalars.

+
+
Parameters:
+
    +

  • a (array) – Input array or scalar.

    • +

  • b (array) – Input array or scalar.

    • +
+
+
Returns:
+

The difference a - b.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.sum.html b/docs/build/html/python/_autosummary/mlx.core.sum.html new file mode 100644 index 000000000..23f5ad6ce --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.sum.html @@ -0,0 +1,254 @@ + + + + + + + mlx.core.sum — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.sum

+
+
+mlx.core.sum(array: array, /, axis: Union[None, int, List[int]] = None, keepdims: bool = False, *, stream: Union[None, Stream, Device] = None) → array
+

Sum reduce the array over the given axes.

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • axis (int or list(int), optional) – Optional axis or +axes to reduce over. If unspecified this defaults +to reducing over the entire array.

    • +

  • keepdims (bool, optional) – Keep reduced axes as +singleton dimensions, defaults to False.

    • +
+
+
Returns:
+

The output array with the corresponding axes reduced.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.take.html b/docs/build/html/python/_autosummary/mlx.core.take.html new file mode 100644 index 000000000..51864ae3a --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.take.html @@ -0,0 +1,256 @@ + + + + + + + mlx.core.take — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.take

+
+
+mlx.core.take(a: array, /, indices: array, axis: Optional[int] = None, *, stream: Union[None, Stream, Device] = None) → array
+

Take elements along an axis.

+

The elements are taken from indices along the specified axis. +If the axis is not specified the array is treated as a flattened +1-D array prior to performing the take.

+

As an example, if the axis=1 this is equialent to a[:, indices, ...].

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • indices (array) – Input array with integral type.

    • +

  • axis (int, optional) – Axis along which to perform the take. If unspecified +the array is treated as a flattened 1-D vector.

    • +
+
+
Returns:
+

The indexed values of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.take_along_axis.html b/docs/build/html/python/_autosummary/mlx.core.take_along_axis.html new file mode 100644 index 000000000..b566370f4 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.take_along_axis.html @@ -0,0 +1,254 @@ + + + + + + + mlx.core.take_along_axis — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.take_along_axis

+
+
+mlx.core.take_along_axis(a: array, /, indices: array, axis: Optional[int], *, stream: Union[None, Stream, Device] = None) → array
+

Take values along an axis at the specified indices.

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • indices (array) – Indices array. These should be broadcastable with +the input array excluding the axis dimension.

    • +

  • axis (int or None) – Axis in the input to take the values from. If +axis == None the array is flattened to 1D prior to the indexing +operation.

    • +
+
+
Returns:
+

The output array with the specified shape and values.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.tan.html b/docs/build/html/python/_autosummary/mlx.core.tan.html new file mode 100644 index 000000000..66551ebd6 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.tan.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.tan — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.tan

+
+
+mlx.core.tan(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise tangent.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The tangent of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.tanh.html b/docs/build/html/python/_autosummary/mlx.core.tanh.html new file mode 100644 index 000000000..a18c2a58e --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.tanh.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.tanh — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.tanh

+
+
+mlx.core.tanh(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

Element-wise hyperbolic tangent.

+
+
Parameters:
+

a (array) – Input array.

+
+
Returns:
+

The hyperbolic tangent of a.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.transpose.html b/docs/build/html/python/_autosummary/mlx.core.transpose.html new file mode 100644 index 000000000..7d46fda97 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.transpose.html @@ -0,0 +1,251 @@ + + + + + + + mlx.core.transpose — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.transpose

+
+
+mlx.core.transpose(a: array, /, axes: Optional[List[int]] = None, *, stream: Union[None, Stream, Device] = None) → array
+

Transpose the dimensions of the array.

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • axes (list(int), optional) – Specifies the source axis for each axis +in the new array. The default is to reverse the axes.

    • +
+
+
Returns:
+

The transposed array.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.value_and_grad.html b/docs/build/html/python/_autosummary/mlx.core.value_and_grad.html new file mode 100644 index 000000000..70880831c --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.value_and_grad.html @@ -0,0 +1,204 @@ + + + + + + + mlx.core.value_and_grad — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.value_and_grad

+
+
+mlx.core.value_and_grad(fun: function, argnums: Union[int, List[int]] = 0) → cpp_function
+

Returns a function which computes the value and gradient of fun.

+

The function passed to value_and_grad() should return either +a scalar loss or a tuple in which the first element is a scalar +loss and the remaining elements can be anything.

+
import mlx.core as mx
+
+def mse(params, inputs, targets):
+    outputs = forward(params, inputs)
+    lvalue = (outputs - targets).square().mean()
+    return lvalue
+
+# Returns lvalue, dlvalue/dparams
+lvalue, grads = mx.value_and_grad(mse)
+
+def lasso(params, inputs, targets, a=1.0, b=1.0):
+    outputs = forward(params, inputs)
+    mse = (outputs - targets).square().mean()
+    l1 = mx.abs(outputs - targets).mean()
+
+    loss = a*mse + b*l1
+
+    return loss, mse, l1
+
+(loss, mse, l1), grads = mx.value_and_grad(lasso)
+
+
+
+
Parameters:
+
    +

  • fun (function) – A function which takes a variable number of +array or trees of array and returns +a scalar output array or a tuple the first element +of which should be a scalar array.

    • +

  • argnums (int or list(int), optional) – Specify the index (or indices) +of the arguments of fun to compute the gradient with respect to. If +unspecified this defaults to 0 indicating fun’s first argument.

    • +
+
+
Returns:
+

A function which returns a tuple where the first element +is the output of fun and the second element is the gradients w.r.t. +the loss.

+
+
Return type:
+

function

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.var.html b/docs/build/html/python/_autosummary/mlx.core.var.html new file mode 100644 index 000000000..fdd4ee50d --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.var.html @@ -0,0 +1,256 @@ + + + + + + + mlx.core.var — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.var

+
+
+mlx.core.var(a: array, /, axis: Union[None, int, List[int]] = None, keepdims: bool = False, ddof: int = 0, *, stream: Union[None, Stream, Device] = None) → array
+

Compute the variance(s) over the given axes.

+
+
Parameters:
+
    +

  • a (array) – Input array.

    • +

  • axis (int or list(int), optional) – Optional axis or +axes to reduce over. If unspecified this defaults +to reducing over the entire array.

    • +

  • keepdims (bool, optional) – Keep reduced axes as +singleton dimensions, defaults to False.

    • +

  • ddof (int, optional) – The divisor to compute the variance +is N - ddof, defaults to 0.

    • +
+
+
Returns:
+

The output array of variances.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.vjp.html b/docs/build/html/python/_autosummary/mlx.core.vjp.html new file mode 100644 index 000000000..5c9a95bdd --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.vjp.html @@ -0,0 +1,180 @@ + + + + + + + mlx.core.vjp — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.vjp

+
+
+mlx.core.vjp(fun: function, primals: List[array], cotangents: List[array]) → Tuple[List[array], List[array]]
+

Compute the vector-Jacobian product.

+

Computes the product of the cotangents with the Jacobian of a +function fun evaluated at primals.

+
+
Parameters:
+
    +

  • fun (function) – A function which takes a variable number of array +and returns a single array or list of array.

    • +

  • primals (list(array)) – A list of array at which to +evaluate the Jacobian.

    • +

  • cotangents (list(array)) – A list of array which are the +“vector” in the vector-Jacobian product. The cotangents should be the +same in number, shape, and type as the outputs of fun.

    • +
+
+
Returns:
+

A list of the vector-Jacobian products which +is the same in number, shape, and type of the outputs of fun.

+
+
Return type:
+

list(array)

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.vmap.html b/docs/build/html/python/_autosummary/mlx.core.vmap.html new file mode 100644 index 000000000..8acbeeed7 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.vmap.html @@ -0,0 +1,181 @@ + + + + + + + mlx.core.vmap — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.vmap

+
+
+mlx.core.vmap(fun: function, in_axes: object = 0, out_axes: object = 0) → cpp_function
+

Returns a vectorized version of fun.

+
+
Parameters:
+
    +

  • fun (function) – A function which takes a variable number of +array or a tree of array and returns +a variable number of array or a tree of array.

    • +

  • in_axes (int, optional) – An integer or a valid prefix tree of the +inputs to fun where each node specifies the vmapped axis. If +the value is None then the corresponding input(s) are not vmapped. +Defaults to 0.

    • +

  • out_axes (int, optional) – An integer or a valid prefix tree of the +outputs of fun where each node specifies the vmapped axis. If +the value is None then the corresponding outputs(s) are not vmapped. +Defaults to 0.

    • +
+
+
Returns:
+

The vectorized function.

+
+
Return type:
+

function

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.where.html b/docs/build/html/python/_autosummary/mlx.core.where.html new file mode 100644 index 000000000..bb2498601 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.where.html @@ -0,0 +1,253 @@ + + + + + + + mlx.core.where — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.where

+
+
+mlx.core.where(condition: Union[bool, int, float, complex, array], x: Union[bool, int, float, complex, array], y: Union[bool, int, float, complex, array], /, *, stream: Union[None, Stream, Device] = None) → array
+

Select from x or y according to condition.

+

The condition and input arrays must be the same shape or broadcastable +with each another.

+
+
Parameters:
+
    +

  • condition (array) – The condition array.

    • +

  • x (array) – The input selected from where condition is True.

    • +

  • y (array) – The input selected from where condition is False.

    • +
+
+
Returns:
+

The output containing elements selected from x and y.

+
+
Return type:
+

result (array)

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.zeros.html b/docs/build/html/python/_autosummary/mlx.core.zeros.html new file mode 100644 index 000000000..4f998a038 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.zeros.html @@ -0,0 +1,251 @@ + + + + + + + mlx.core.zeros — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.zeros

+
+
+mlx.core.zeros(shape: Union[int, List[int]], dtype: Optional[Dtype] = None, *, stream: Union[None, Stream, Device] = None) → array
+

Construct an array of zeros.

+
+
Parameters:
+
    +

  • shape (int or list(int)) – The shape of the output array.

    • +

  • dtype (Dtype, optional) – Data type of the output array. If +unspecified the output type defaults to float32.

    • +
+
+
Returns:
+

The array of zeros with the specified shape.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.core.zeros_like.html b/docs/build/html/python/_autosummary/mlx.core.zeros_like.html new file mode 100644 index 000000000..56887d8f6 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.core.zeros_like.html @@ -0,0 +1,247 @@ + + + + + + + mlx.core.zeros_like — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.core.zeros_like

+
+
+mlx.core.zeros_like(a: array, /, *, stream: Union[None, Stream, Device] = None) → array
+

An array of zeros like the input.

+
+
Parameters:
+

a (array) – The input to take the shape and type from.

+
+
Returns:
+

The output array filled with zeros.

+
+
Return type:
+

array

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.nn.Conv1d.html b/docs/build/html/python/_autosummary/mlx.nn.Conv1d.html new file mode 100644 index 000000000..6cd3d8df9 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.nn.Conv1d.html @@ -0,0 +1,199 @@ + + + + + + + mlx.nn.Conv1d — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.nn.Conv1d

+
+
+class mlx.nn.Conv1d(in_channels: int, out_channels: int, kernel_size: int, stride: int = 1, padding: int = 0, bias: bool = True)
+

Applies a 1-dimensional convolution over the multi-channel input sequence.

+
+
The channels are expected to be last i.e. the input shape should be NLC where:
    +

  • N is the batch dimension

    • +

  • L is the sequence length

    • +

  • C is the number of input channels

    • +
+
+
+
+
Parameters:
+
    +

  • in_channels (int) – The number of input channels

    • +

  • out_channels (int) – The number of output channels

    • +

  • kernel_size (int) – The size of the convolution filters

    • +

  • stride (int, optional) – The stride when applying the filter. +Default: 1.

    • +

  • padding (int, optional) – How many positions to 0-pad the input with. +Default: 0.

    • +

  • bias (bool, optional) – If True add a learnable bias to the output. +Default: True

    • +
+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.nn.Conv2d.html b/docs/build/html/python/_autosummary/mlx.nn.Conv2d.html new file mode 100644 index 000000000..2aa681f4d --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.nn.Conv2d.html @@ -0,0 +1,200 @@ + + + + + + + mlx.nn.Conv2d — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.nn.Conv2d

+
+
+class mlx.nn.Conv2d(in_channels: int, out_channels: int, kernel_size: Union[int, tuple], stride: Union[int, tuple] = 1, padding: Union[int, tuple] = 0, bias: bool = True)
+

Applies a 2-dimensional convolution over the multi-channel input image.

+
+
The channels are expected to be last i.e. the input shape should be NHWC where:
    +

  • N is the batch dimension

    • +

  • H is the input image height

    • +

  • W is the input image width

    • +

  • C is the number of input channels

    • +
+
+
+
+
Parameters:
+
    +

  • in_channels (int) – The number of input channels.

    • +

  • out_channels (int) – The number of output channels.

    • +

  • kernel_size (int or tuple) – The size of the convolution filters.

    • +

  • stride (int or tuple, optional) – The size of the stride when +applying the filter. Default: 0.

    • +

  • padding (int or tuple, optional) – How many positions to 0-pad +the input with. Default: 0.

    • +

  • bias (bool, optional) – If True add a learnable bias to the +output. Default: True

    • +
+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.nn.Embedding.html b/docs/build/html/python/_autosummary/mlx.nn.Embedding.html new file mode 100644 index 000000000..a82b55f9e --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.nn.Embedding.html @@ -0,0 +1,187 @@ + + + + + + + mlx.nn.Embedding — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.nn.Embedding

+
+
+class mlx.nn.Embedding(num_embeddings: int, dims: int)
+

Implements a simple lookup table that maps each input integer to a +high-dimensional vector.

+

Typically used to embed discrete tokens for processing by neural networks.

+
+
Parameters:
+
    +

  • num_embeddings (int) – How many possible discrete tokens can we embed. +Usually called the vocabulary size.

    • +

  • dims (int) – The dimensionality of the embeddings.

    • +
+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.nn.GELU.html b/docs/build/html/python/_autosummary/mlx.nn.GELU.html new file mode 100644 index 000000000..61f9038ff --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.nn.GELU.html @@ -0,0 +1,192 @@ + + + + + + + mlx.nn.GELU — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.nn.GELU

+
+
+class mlx.nn.GELU(approx='none')
+

Applies the Gaussian Error Linear Units.

+
+\[\textrm{GELU}(x) = x * \Phi(x)\]
+

where \(\Phi(x)\) is the Gaussian CDF.

+

However, if approx is set to ‘precise’ or ‘fast’ it applies

+
+\[\begin{split}\textrm{GELUApprox}(x) &= x * \sigma\left(1.60033 * x \left(1 + 0.0433603 * x^2\right)\right) \\ +\textrm{GELUFast}(x) &= x * \sigma\left(1.773 * x\right)\end{split}\]
+

respectively.

+

See gelu(), gelu_approx() and gelu_fast_approx() for the +functional equivalents and information regarding error bounds.

+
+
Parameters:
+

approx ('none' | 'precise' | 'fast') – Which approximation to gelu to use if any.

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.nn.GroupNorm.html b/docs/build/html/python/_autosummary/mlx.nn.GroupNorm.html new file mode 100644 index 000000000..6e4324e00 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.nn.GroupNorm.html @@ -0,0 +1,201 @@ + + + + + + + mlx.nn.GroupNorm — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.nn.GroupNorm

+
+
+class mlx.nn.GroupNorm(num_groups: int, dims: int, eps: float = 1e-05, affine: bool = True, pytorch_compatible: bool = False)
+

Applies Group Normalization [1] to the inputs.

+

Computes the same normalization as layer norm, namely

+
+\[y = \frac{x - E[x]}{\sqrt{Var[x]} + \epsilon} \gamma + \beta,\]
+

where \(\gamma\) and \(\beta\) are learned per feature dimension +parameters initialized at 1 and 0 respectively. However, the mean and +variance are computed over the spatial dimensions and each group of +features. In particular, the input is split into num_groups accross the +feature dimension.

+

The feature dimension is assumed to be the last dimension and the dimensions +that precede it (except the first) are considered the spatial dimensions.

+

[1]: https://arxiv.org/abs/1803.08494

+
+
Parameters:
+
    +

  • num_groups (int) – Number of groups to separate the features into

    • +

  • dims (int) – The feature dimensions of the input to normalize over

    • +

  • eps (float) – A small additive constant for numerical stability

    • +

  • affine (bool) – If True learn an affine transform to apply after the +normalization.

    • +

  • pytorch_compatible (bool) – If True perform the group normalization in +the same order/grouping as PyTorch.

    • +
+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.nn.LayerNorm.html b/docs/build/html/python/_autosummary/mlx.nn.LayerNorm.html new file mode 100644 index 000000000..69f44241c --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.nn.LayerNorm.html @@ -0,0 +1,193 @@ + + + + + + + mlx.nn.LayerNorm — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.nn.LayerNorm

+
+
+class mlx.nn.LayerNorm(dims: int, eps: float = 1e-05, affine: bool = True)
+

Applies layer normalization [1] on the inputs.

+

Computes

+
+\[y = \frac{x - E[x]}{\sqrt{Var[x]} + \epsilon} \gamma + \beta,\]
+

where \(\gamma\) and \(\beta\) are learned per feature dimension +parameters initialized at 1 and 0 respectively.

+

[1]: https://arxiv.org/abs/1607.06450

+
+
Parameters:
+
    +

  • dims (int) – The feature dimension of the input to normalize over

    • +

  • eps (float) – A small additive constant for numerical stability

    • +

  • affine (bool) – If True learn an affine transform to apply after the +normalization

    • +
+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.nn.Linear.html b/docs/build/html/python/_autosummary/mlx.nn.Linear.html new file mode 100644 index 000000000..18a60a395 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.nn.Linear.html @@ -0,0 +1,185 @@ + + + + + + + mlx.nn.Linear — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.nn.Linear

+
+
+class mlx.nn.Linear(input_dims: int, output_dims: int, bias: bool = True)
+

Applies an affine transformation to the input.

+
+
Parameters:
+
    +

  • input_dims (int) – The dimensionality of the input features

    • +

  • output_dims (int) – The dimensionality of the output features

    • +

  • bias (bool) – If set to False then the layer will not use a bias

    • +
+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.nn.MultiHeadAttention.html b/docs/build/html/python/_autosummary/mlx.nn.MultiHeadAttention.html new file mode 100644 index 000000000..c05e84545 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.nn.MultiHeadAttention.html @@ -0,0 +1,198 @@ + + + + + + + mlx.nn.MultiHeadAttention — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.nn.MultiHeadAttention

+
+
+class mlx.nn.MultiHeadAttention(dims: int, num_heads: int, query_input_dims: Optional[int] = None, key_input_dims: Optional[int] = None, value_input_dims: Optional[int] = None, value_dims: Optional[int] = None, value_output_dims: Optional[int] = None)
+

Implements the scaled dot product attention with multiple heads.

+

Given inputs for queries, keys and values the MultiHeadAttention produces +new values by aggregating information from the input values according to +the similarities of the input queries and keys.

+

All inputs as well as the output are lineary projected without biases.

+

MultiHeadAttention also expects an additive attention mask that should be +broadcastable with (batch, num_heads, # queries, # keys). The mask should +have -inf or very negative numbers to the positions that should not be +attended to.

+
+
Parameters:
+
    +

  • dims (int) – The model dimensions. If no other dims are provided then +dims is used for queries, keys, values and the output.

    • +

  • num_heads (int) – How many attention heads to use

    • +

  • query_input_dims (int, optional) – The input dimensions of the queries (default: dims).

    • +

  • key_input_dims (int, optional) – The input dimensions of the keys (default: dims).

    • +

  • value_input_dims (int, optional) – The input dimensions of the values (default: key_input_dims).

    • +

  • value_dims (int, optional) – The dimensions of the values after the projection (default: dims).

    • +

  • value_output_dims (int, optional) – The dimensions the new values will be projected to (default: dims).

    • +
+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.nn.RMSNorm.html b/docs/build/html/python/_autosummary/mlx.nn.RMSNorm.html new file mode 100644 index 000000000..44a841911 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.nn.RMSNorm.html @@ -0,0 +1,191 @@ + + + + + + + mlx.nn.RMSNorm — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.nn.RMSNorm

+
+
+class mlx.nn.RMSNorm(dims: int, eps: float = 1e-05)
+

Applies Root Mean Square normalization [1] to the inputs.

+

Computes

+
+\[y = \frac{x}{\sqrt{E[x^2] + \epsilon}} \gamma\]
+

where \(\gamma\) is a learned per feature dimension parameter initialized at +1.

+

[1]: https://arxiv.org/abs/1910.07467

+
+
Parameters:
+
    +

  • dims (int) – The feature dimension of the input to normalize over

    • +

  • eps (float) – A small additive constant for numerical stability

    • +
+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.nn.ReLU.html b/docs/build/html/python/_autosummary/mlx.nn.ReLU.html new file mode 100644 index 000000000..3f61f9e2c --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.nn.ReLU.html @@ -0,0 +1,177 @@ + + + + + + + mlx.nn.ReLU — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.nn.ReLU

+
+
+class mlx.nn.ReLU
+

Applies the Rectified Linear Unit.

+

Simply mx.maximum(x, 0).

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.nn.RoPE.html b/docs/build/html/python/_autosummary/mlx.nn.RoPE.html new file mode 100644 index 000000000..0e95f2795 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.nn.RoPE.html @@ -0,0 +1,190 @@ + + + + + + + mlx.nn.RoPE — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.nn.RoPE

+
+
+class mlx.nn.RoPE(dims: int, traditional: bool = False)
+

Implements the rotary positional encoding [1].

+

The traditional implementation rotates consecutive pairs of elements in the +feature dimension while the default implementation rotates pairs with +stride half the feature dimensions for efficiency.

+

[1]: https://arxiv.org/abs/2104.09864

+
+
Parameters:
+
    +

  • dims (int) – The feature dimensions to be rotated. If the input feature +is larger than dims then the rest is left unchanged.

    • +

  • traditional (bool) – If set to True choose the traditional +implementation which is slightly less efficient.

    • +
+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.nn.Sequential.html b/docs/build/html/python/_autosummary/mlx.nn.Sequential.html new file mode 100644 index 000000000..fe64b9420 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.nn.Sequential.html @@ -0,0 +1,184 @@ + + + + + + + mlx.nn.Sequential — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.nn.Sequential

+
+
+class mlx.nn.Sequential(*modules)
+

A layer that calls the passed callables in order.

+

We can pass either modules or plain callables to the Sequential module. If +our functions have learnable parameters they should be implemented as +nn.Module instances.

+
+
Parameters:
+

modules (tuple of Callables) – The modules to call in order

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.nn.SiLU.html b/docs/build/html/python/_autosummary/mlx.nn.SiLU.html new file mode 100644 index 000000000..e4df5fb3b --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.nn.SiLU.html @@ -0,0 +1,179 @@ + + + + + + + mlx.nn.SiLU — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.nn.SiLU

+
+
+class mlx.nn.SiLU
+

Applies the Sigmoid Linear Unit.

+

Applies \(x \sigma(x)\) element wise, where \(\sigma(\cdot)\) is +the logistic sigmoid.

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.nn.value_and_grad.html b/docs/build/html/python/_autosummary/mlx.nn.value_and_grad.html new file mode 100644 index 000000000..8daef1f8d --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.nn.value_and_grad.html @@ -0,0 +1,174 @@ + + + + + + + mlx.nn.value_and_grad — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.nn.value_and_grad

+
+
+mlx.nn.value_and_grad(model: Module, fn: Callable)
+

Transform the passed function fn to a function that computes the +gradients of fn wrt the model’s trainable parameters and also its +value.

+
+
Parameters:
+
    +

  • model (Module) – The model whose trainable parameters to compute +gradients for

    • +

  • fn (Callable) – The scalar function to compute gradients for

    • +
+
+
Returns:
+

A callable that returns the value of fn and the gradients wrt the +trainable parameters of model

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.optimizers.Adam.html b/docs/build/html/python/_autosummary/mlx.optimizers.Adam.html new file mode 100644 index 000000000..4572662f5 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.optimizers.Adam.html @@ -0,0 +1,176 @@ + + + + + + + mlx.optimizers.Adam — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.optimizers.Adam

+
+
+class mlx.optimizers.Adam(learning_rate: float, betas: List[float] = [0.9, 0.999], eps: float = 1e-08)
+

Implementation of the Adam optimizer [1].

+

Our Adam implementation follows the original paper and omits the bias +correction in the first and second moment estimates. In detail,

+
+\[\begin{split}m_{t+1} &= \beta_1 m_t + (1 - \beta_1) g_t \\ +v_{t+1} &= \beta_2 v_t + (1 - \beta_2) g_t^2 \\ +w_{t+1} &= w_t - \lambda \frac{m_{t+1}}{\sqrt{v_{t+1} + \epsilon}}\end{split}\]
+

[1]: Kingma, D.P. and Ba, J., 2015. Adam: A method for stochastic +optimization. ICLR 2015.

+

Methods

+ + + + + + + + + +

__init__(learning_rate[, betas, eps])

apply_single(gradient, parameter, state)

Performs the Adam parameter update and stores \(v\) and \(m\) in the optimizer state.

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.optimizers.Optimizer.html b/docs/build/html/python/_autosummary/mlx.optimizers.Optimizer.html new file mode 100644 index 000000000..4442f5d7a --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.optimizers.Optimizer.html @@ -0,0 +1,185 @@ + + + + + + + mlx.optimizers.Optimizer — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.optimizers.Optimizer

+
+
+class mlx.optimizers.Optimizer
+

The base class for all optimizers. It allows us to implement an +optimizer on a per-parameter basis and apply it to a parameter tree.

+
+
+state
+

It holds the optimizer’s state dictionary.

+
+
Type:
+

OptimizerState

+
+
+
+ +

Methods

+ + + + + + + + + + + + + + + +

__init__()

apply_gradients(gradients, model)

Apply the gradients to the parameters and return the updated parameters.

apply_single(gradient, parameter, state)

To be extended by the children classes to implement each optimizer's update.

update(model, gradients)

Apply the gradients to the parameters of the model and update the model with the new parameters.

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.optimizers.OptimizerState.html b/docs/build/html/python/_autosummary/mlx.optimizers.OptimizerState.html new file mode 100644 index 000000000..dc561f342 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.optimizers.OptimizerState.html @@ -0,0 +1,172 @@ + + + + + + + mlx.optimizers.OptimizerState — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.optimizers.OptimizerState

+
+
+class mlx.optimizers.OptimizerState
+

The optimizer state implements a recursively defined +collections.defaultdict, namely a missing key in an optimizer +state is an OptimizerState.

+
+

Note

+

OptimizerState.get() in contrast to a normal dictionary also sets +the key to the default value if the key was not present in the +dictionary.

+
+

Methods

+ + + + + + +

get(key, default)

If key doesn't exist set its value to default and then return it.

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.optimizers.SGD.html b/docs/build/html/python/_autosummary/mlx.optimizers.SGD.html new file mode 100644 index 000000000..f07802663 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.optimizers.SGD.html @@ -0,0 +1,180 @@ + + + + + + + mlx.optimizers.SGD — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.optimizers.SGD

+
+
+class mlx.optimizers.SGD(learning_rate: float, momentum: float = 0.0)
+

Stochastic gradient descent optimizer.

+

Updates a parameter \(w\) with a gradient \(g\) as follows

+
+\[\begin{split}v_{t+1} &= \mu v_t + (1 - \mu) g_t \\ +w_{t+1} &= w_t - \lambda v_{t+1}\end{split}\]
+
+
Parameters:
+
    +

  • learning_rate (float) – The learning \(\lambda\) for the update

    • +

  • momentum (float) – The momentum strength \(\mu\)

    • +
+
+
+

Methods

+ + + + + + + + + +

__init__(learning_rate[, momentum])

apply_single(gradient, parameter, state)

Performs the SGD parameter update and stores \(v\) in the optimizer state.

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.utils.tree_flatten.html b/docs/build/html/python/_autosummary/mlx.utils.tree_flatten.html new file mode 100644 index 000000000..6de1719c4 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.utils.tree_flatten.html @@ -0,0 +1,187 @@ + + + + + + + mlx.utils.tree_flatten — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.utils.tree_flatten

+
+
+mlx.utils.tree_flatten(tree, prefix='', is_leaf=None)
+

Flattens a python tree to a list of key, value tuples.

+

The keys are using the dot notation to define trees of arbitrary depth and +complexity.

+
from mlx.utils import tree_flatten
+
+print(tree_flatten([[[0]]]))
+# [("0.0.0", 0)]
+
+print(tree_flatten([[[0]]], ".hello"))
+# [("hello.0.0.0", 0)]
+
+
+
+

Note

+

Dictionaries should have keys that are valid python identifiers.

+
+
+
Parameters:
+
    +

  • tree (Any) – The python tree to be flattened.

    • +

  • prefix (str) – A prefix to use for the keys. The first character is +always discarded.

    • +

  • is_leaf (Callable) – An optional callable that returns True if the +passed object is considered a leaf or False otherwise.

    • +
+
+
Returns:
+

The flat representation of the python tree.

+
+
Return type:
+

List[Tuple[str, Any]]

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.utils.tree_map.html b/docs/build/html/python/_autosummary/mlx.utils.tree_map.html new file mode 100644 index 000000000..22e240d83 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.utils.tree_map.html @@ -0,0 +1,183 @@ + + + + + + + mlx.utils.tree_map — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.utils.tree_map

+
+
+mlx.utils.tree_map(fn, tree, *rest)
+

Applies fn to the leaves of the python tree tree and +returns a new collection with the results.

+

If rest is provided, every item is assumed to be a superset of tree +and the corresponding leaves are provided as extra positional arguments to +fn. In that respect, tree_map() is closer to itertools.starmap() +than to map().

+
import mlx.nn as nn
+from mlx.utils import tree_map
+
+model = nn.Linear(10, 10)
+print(model.parameters().keys())
+# dict_keys(['weight', 'bias'])
+
+# square the parameters
+model.update(tree_map(lambda x: x*x, model.parameters()))
+
+
+
+
Parameters:
+
    +

  • fn (Callable) – The function that processes the leaves of the tree

    • +

  • tree (Any) – The main python tree that will be iterated upon

    • +

  • rest (Tuple[Any]) – Extra trees to be iterated together with tree

    • +
+
+
Returns:
+

A python tree with the new values returned by fn.

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary/mlx.utils.tree_unflatten.html b/docs/build/html/python/_autosummary/mlx.utils.tree_unflatten.html new file mode 100644 index 000000000..ff771bed8 --- /dev/null +++ b/docs/build/html/python/_autosummary/mlx.utils.tree_unflatten.html @@ -0,0 +1,171 @@ + + + + + + + mlx.utils.tree_unflatten — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.utils.tree_unflatten

+
+
+mlx.utils.tree_unflatten(tree)
+

Recreate a python tree from its flat representation.

+
from mlx.utils import tree_unflatten
+
+d = tree_unflatten([("hello.world", 42)])
+print(d)
+# {"hello": {"world": 42}}
+
+
+
+
Parameters:
+

tree (List[Tuple[str, Any]]) – The flat representation of a python tree. +For instance as returned by tree_flatten().

+
+
Returns:
+

A python tree.

+
+
+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary_functions/mlx.nn.gelu.html b/docs/build/html/python/_autosummary_functions/mlx.nn.gelu.html new file mode 100644 index 000000000..6f267023a --- /dev/null +++ b/docs/build/html/python/_autosummary_functions/mlx.nn.gelu.html @@ -0,0 +1,182 @@ + + + + + + + mlx.nn.gelu — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.nn.gelu

+
+
+class mlx.nn.gelu(x)
+

Applies the Gaussian Error Linear Units function.

+
+\[\textrm{GELU}(x) = x * \Phi(x)\]
+

where \(\Phi(x)\) is the Gaussian CDF.

+

See also gelu_approx() and gelu_fast_approx() for faster +approximations.

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary_functions/mlx.nn.gelu_approx.html b/docs/build/html/python/_autosummary_functions/mlx.nn.gelu_approx.html new file mode 100644 index 000000000..b693ea627 --- /dev/null +++ b/docs/build/html/python/_autosummary_functions/mlx.nn.gelu_approx.html @@ -0,0 +1,183 @@ + + + + + + + mlx.nn.gelu_approx — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.nn.gelu_approx

+
+
+class mlx.nn.gelu_approx(x)
+

An approximation to Gaussian Error Linear Unit.

+

See gelu() for the exact computation.

+

This function approximates gelu with a maximum absolute error \(< +0.0003\) in the range \([-6, 6]\) using the following

+
+\[x = x \sigma\left(1.60033 x \left(1 + 0.0433603 x^2\right)\right)\]
+

where \(\sigma(\cdot)\) is the logistic sigmoid.

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary_functions/mlx.nn.gelu_fast_approx.html b/docs/build/html/python/_autosummary_functions/mlx.nn.gelu_fast_approx.html new file mode 100644 index 000000000..5491bfce9 --- /dev/null +++ b/docs/build/html/python/_autosummary_functions/mlx.nn.gelu_fast_approx.html @@ -0,0 +1,183 @@ + + + + + + + mlx.nn.gelu_fast_approx — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.nn.gelu_fast_approx

+
+
+class mlx.nn.gelu_fast_approx(x)
+

A fast approximation to Gaussian Error Linear Unit.

+

See gelu() for the exact computation.

+

This function approximates gelu with a maximum absolute error \(< +0.015\) in the range \([-6, 6]\) using the following

+
+\[x = x \sigma\left(1.773 x\right)\]
+

where \(\sigma(\cdot)\) is the logistic sigmoid.

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary_functions/mlx.nn.relu.html b/docs/build/html/python/_autosummary_functions/mlx.nn.relu.html new file mode 100644 index 000000000..e208fab6b --- /dev/null +++ b/docs/build/html/python/_autosummary_functions/mlx.nn.relu.html @@ -0,0 +1,177 @@ + + + + + + + mlx.nn.relu — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.nn.relu

+
+
+class mlx.nn.relu(x)
+

Applies the Rectified Linear Unit.

+

Simply mx.maximum(x, 0).

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/_autosummary_functions/mlx.nn.silu.html b/docs/build/html/python/_autosummary_functions/mlx.nn.silu.html new file mode 100644 index 000000000..314014b51 --- /dev/null +++ b/docs/build/html/python/_autosummary_functions/mlx.nn.silu.html @@ -0,0 +1,179 @@ + + + + + + + mlx.nn.silu — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.nn.silu

+
+
+class mlx.nn.silu(x)
+

Applies the Sigmoid Linear Unit.

+

Applies \(x \sigma(x)\) element wise, where \(\sigma(\cdot)\) is +the logistic sigmoid.

+
+ +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/array.html b/docs/build/html/python/array.html new file mode 100644 index 000000000..722e51337 --- /dev/null +++ b/docs/build/html/python/array.html @@ -0,0 +1,289 @@ + + + + + + + Array — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Array

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

array

An N-dimensional array object.

array.astype(self, dtype[, stream])

Cast the array to a specified type.

array.item(self)

Access the value of a scalar array.

array.tolist(self)

Convert the array to a Python list.

array.dtype

The array's Dtype.

array.ndim

The array's dimension.

array.shape

The shape of the array as a Python list.

array.size

Number of elments in the array.

Dtype

An object to hold the type of a array.

array.abs(self, *[, stream])

See abs().

array.all(self[, axis, keepdims, stream])

See all().

array.any(self[, axis, keepdims, stream])

See any().

array.argmax(self[, axis, keepdims, stream])

See argmax().

array.argmin(self[, axis, keepdims, stream])

See argmin().

array.cos(self, *[, stream])

See cos().

array.dtype

The array's Dtype.

array.exp(self, *[, stream])

See exp().

array.log(self, *[, stream])

See log().

array.log1p(self, *[, stream])

See log1p().

array.logsumexp(self[, axis, keepdims, stream])

See logsumexp().

array.max(self[, axis, keepdims, stream])

See max().

array.mean(self[, axis, keepdims, stream])

See mean().

array.min(self[, axis, keepdims, stream])

See min().

array.prod(self[, axis, keepdims, stream])

See prod().

array.reciprocal(self, *[, stream])

See reciprocal().

array.reshape(self, *args[, stream])

Equivalent to reshape() but the shape can be passed either as a tuple or as separate arguments.

array.rsqrt(self, *[, stream])

See rsqrt().

array.sin(self, *[, stream])

See sin().

array.split(self, indices_or_sections[, ...])

See split().

array.sqrt(self, *[, stream])

See sqrt().

array.square(self, *[, stream])

See square().

array.sum(self[, axis, keepdims, stream])

See sum().

array.transpose(self, *args[, stream])

Equivalent to transpose() but the axes can be passed either as a tuple or as separate arguments.

array.T

Equivalent to calling self.transpose() with no arguments.

array.var(self[, axis, keepdims, ddof, stream])

See var().

+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/data_types.html b/docs/build/html/python/data_types.html new file mode 100644 index 000000000..9c04ae830 --- /dev/null +++ b/docs/build/html/python/data_types.html @@ -0,0 +1,200 @@ + + + + + + + Data Types — MLX 0.0.0 documentation + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Data Types

+

The default floating point type is float32 and the default integer type is +int32. The table below shows supported values for Dtype.

+ + +++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Supported Data Types

Type

Bytes

Description

bool_

1

Boolean (True, False) data type

uint8

1

8-bit unsigned integer

uint16

2

16-bit unsigned integer

uint32

4

32-bit unsigned integer

uint32

8

32-bit unsigned integer

int8

1

8-bit signed integer

int16

2

16-bit signed integer

int32

4

32-bit signed integer

int64

8

64-bit signed integer

float16

2

16-bit float, only available with ARM C language extensions

float32

4

32-bit float

+
+ + +
+
+
+ +
+ +
+

© Copyright 2023, MLX Contributors.

+
+ + Built with Sphinx using a + theme + provided by Read the Docs. + + +
+
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/devices_and_streams.html b/docs/build/html/python/devices_and_streams.html new file mode 100644 index 000000000..cd5abf0a8 --- /dev/null +++ b/docs/build/html/python/devices_and_streams.html @@ -0,0 +1,177 @@ + + + + + + + Devices and Streams — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Devices and Streams

+ + + + + + + + + + + + + + + + + + + + + + + + +

Device

default_device()

set_default_device(device)

Stream

default_stream(device)

new_stream(device)

set_default_stream(stream)

+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/fft.html b/docs/build/html/python/fft.html new file mode 100644 index 000000000..334bfc03a --- /dev/null +++ b/docs/build/html/python/fft.html @@ -0,0 +1,197 @@ + + + + + + + FFT — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

FFT

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

fft(a[, n, axis, stream])

One dimensional discrete Fourier Transform.

ifft(a[, n, axis, stream])

One dimensional inverse discrete Fourier Transform.

fft2(a[, s, axes, stream])

Two dimensional discrete Fourier Transform.

ifft2(a[, s, axes, stream])

Two dimensional inverse discrete Fourier Transform.

fftn(a[, s, axes, stream])

n-dimensional discrete Fourier Transform.

ifftn(a[, s, axes, stream])

n-dimensional inverse discrete Fourier Transform.

rfft(a[, n, axis, stream])

One dimensional discrete Fourier Transform on a real input.

irfft(a[, n, axis, stream])

The inverse of rfft().

rfft2(a[, s, axes, stream])

Two dimensional real discrete Fourier Transform.

irfft2(a[, s, axes, stream])

The inverse of rfft2().

rfftn(a[, s, axes, stream])

n-dimensional real discrete Fourier Transform.

irfftn(a[, s, axes, stream])

The inverse of rfftn().

+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/nn.html b/docs/build/html/python/nn.html new file mode 100644 index 000000000..30d3ebf4b --- /dev/null +++ b/docs/build/html/python/nn.html @@ -0,0 +1,362 @@ + + + + + + + Neural Networks — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Neural Networks

+

Writing arbitrarily complex neural networks in MLX can be done using only +mlx.core.array and mlx.core.value_and_grad(). However, this requires the +user to write again and again the same simple neural network operations as well +as handle all the parameter state and initialization manually and explicitly.

+

The module mlx.nn solves this problem by providing an intuitive way of +composing neural network layers, initializing their parameters, freezing them +for finetuning and more.

+
+

Quick Start with Neural Networks

+
import mlx.core as mx
+import mlx.nn as nn
+
+class MLP(nn.Module):
+    def __init__(self, in_dims: int, out_dims: int):
+        super().__init__()
+
+        self.layers = [
+            nn.Linear(in_dims, 128),
+            nn.Linear(128, 128),
+            nn.Linear(128, out_dims),
+        ]
+
+    def __call__(self, x):
+        for i, l in enumerate(self.layers):
+            x = mx.maximum(x, 0) if i > 0 else x
+            x = l(x)
+        return x
+
+# The model is created with all its parameters but nothing is initialized
+# yet because MLX is lazily evaluated
+mlp = MLP(2, 10)
+
+# We can access its parameters by calling mlp.parameters()
+params = mlp.parameters()
+print(params["layers"][0]["weight"].shape)
+
+# Printing a parameter will cause it to be evaluated and thus initialized
+print(params["layers"][0])
+
+# We can also force evaluate all parameters to initialize the model
+mx.eval(mlp.parameters())
+
+# A simple loss function.
+# NOTE: It doesn't matter how it uses the mlp model. It currently captures
+#       it from the local scope. It could be a positional argument or a
+#       keyword argument.
+def l2_loss(x, y):
+    y_hat = mlp(x)
+    return (y_hat - y).square().mean()
+
+# Calling `nn.value_and_grad` instead of `mx.value_and_grad` returns the
+# gradient with respect to `mlp.trainable_parameters()`
+loss_and_grad = nn.value_and_grad(mlp, l2_loss)
+
+
+
+
+

The Module Class

+

The workhorse of any neural network library is the Module class. In +MLX the Module class is a container of mlx.core.array or +Module instances. Its main function is to provide a way to +recursively access and update its parameters and those of its +submodules.

+
+

Parameters

+

A parameter of a module is any public member of type mlx.core.array (its +name should not start with _). It can be arbitrarily nested in other +Module instances or lists and dictionaries.

+

Module.parameters() can be used to extract a nested dictionary with all +the parameters of a module and its submodules.

+

A Module can also keep track of “frozen” parameters. +Module.trainable_parameters() returns only the subset of +Module.parameters() that is not frozen. When using +mlx.nn.value_and_grad() the gradients returned will be with respect to these +trainable parameters.

+
+
+

Updating the parameters

+

MLX modules allow accessing and updating individual parameters. However, most +times we need to update large subsets of a module’s parameters. This action is +performed by Module.update().

+
+
+
+

Value and grad

+

Using a Module does not preclude using MLX’s high order function +transformations (mlx.core.value_and_grad(), mlx.core.grad(), etc.). However, +these function transformations assume pure functions, namely the parameters +should be passed as an argument to the function being transformed.

+

There is an easy pattern to achieve that with MLX modules

+
model = ...
+
+def f(params, other_inputs):
+    model.update(params)  # <---- Necessary to make the model use the passed parameters
+    return model(other_inputs)
+
+f(model.trainable_parameters(), mx.zeros((10,)))
+
+
+

However, mlx.nn.value_and_grad() provides precisely this pattern and only +computes the gradients with respect to the trainable parameters of the model.

+

In detail:

+
    +

  • it wraps the passed function with a function that calls Module.update() +to make sure the model is using the provided parameters.

    • +

  • it calls mlx.core.value_and_grad() to transform the function into a function +that also computes the gradients with respect to the passed parameters.

    • +

  • it wraps the returned function with a function that passes the trainable +parameters as the first argument to the function returned by +mlx.core.value_and_grad()

    • +
+ + + + + + +

value_and_grad(model, fn)

Transform the passed function fn to a function that computes the gradients of fn wrt the model's trainable parameters and also its value.

+
+
+

Neural Network Layers

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Embedding(num_embeddings, dims)

Implements a simple lookup table that maps each input integer to a high-dimensional vector.

ReLU()

Applies the Rectified Linear Unit.

GELU([approx])

Applies the Gaussian Error Linear Units.

SiLU()

Applies the Sigmoid Linear Unit.

Linear(input_dims, output_dims[, bias])

Applies an affine transformation to the input.

Conv1d(in_channels, out_channels, kernel_size)

Applies a 1-dimensional convolution over the multi-channel input sequence.

Conv2d(in_channels, out_channels, kernel_size)

Applies a 2-dimensional convolution over the multi-channel input image.

LayerNorm(dims[, eps, affine])

Applies layer normalization [1] on the inputs.

RMSNorm(dims[, eps])

Applies Root Mean Square normalization [1] to the inputs.

GroupNorm(num_groups, dims[, eps, affine, ...])

Applies Group Normalization [1] to the inputs.

RoPE(dims[, traditional])

Implements the rotary positional encoding [1].

MultiHeadAttention(dims, num_heads[, ...])

Implements the scaled dot product attention with multiple heads.

Sequential(*modules)

A layer that calls the passed callables in order.

+

Layers without parameters (e.g. activation functions) are also provided as +simple functions.

+ + + + + + + + + + + + + + + + + + +

gelu(x)

Applies the Gaussian Error Linear Units function.

gelu_approx(x)

An approximation to Gaussian Error Linear Unit.

gelu_fast_approx(x)

A fast approximation to Gaussian Error Linear Unit.

relu(x)

Applies the Rectified Linear Unit.

silu(x)

Applies the Sigmoid Linear Unit.

+
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/nn/module.html b/docs/build/html/python/nn/module.html new file mode 100644 index 000000000..60a6c8165 --- /dev/null +++ b/docs/build/html/python/nn/module.html @@ -0,0 +1,374 @@ + + + + + + + mlx.nn.Module — MLX 0.0.0 documentation + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

mlx.nn.Module

+
+
+class mlx.nn.Module
+

Base class for building neural networks with MLX.

+

All the layers provided in mlx.nn.layers subclass this class and +your models should do the same.

+

A Module can contain other Module instances or mlx.core.array +instances in arbitrary nesting of python lists or dicts. The Module +then allows recursively extracting all the mlx.core.array instances +using mlx.nn.Module.parameters().

+

In addition, the Module has the concept of trainable and non trainable +parameters (called “frozen”). When using mlx.nn.value_and_grad() +the gradients are returned only with respect to the trainable parameters. +All arrays in a module are trainable unless they are added in the “frozen” +set by calling freeze().

+
import mlx.core as mx
+import mlx.nn as nn
+
+class MyMLP(nn.Module):
+    def __init__(self, in_dims: int, out_dims: int, hidden_dims: int = 16):
+        super().__init__()
+
+        self.in_proj = nn.Linear(in_dims, hidden_dims)
+        self.out_proj = nn.Linear(hidden_dims, out_dims)
+
+    def __call__(self, x):
+        x = self.in_proj(x)
+        x = mx.maximum(x, 0)
+        return self.out_proj(x)
+
+model = MyMLP(2, 1)
+
+# All the model parameters are created but since MLX is lazy by
+# default, they are not evaluated yet. Calling `mx.eval` actually
+# allocates memory and initializes the parameters.
+mx.eval(model.parameters())
+
+# Setting a parameter to a new value is as simply as accessing that
+# parameter and assigning a new array to it.
+model.in_proj.weight = model.in_proj.weight * 2
+mx.eval(model.parameters())
+
+
+
+
+apply(map_fn: Callable[[array], array], filter_fn: Optional[Callable[[Module, str, Any], bool]] = None)
+

Map all the parameters using the provided map_fn and immediately +update the module with the mapped parameters.

+

For instance running model.apply(lambda x: x.astype(mx.float16)) +casts all parameters to 16 bit floats.

+
+
Parameters:
+
    +

  • map_fn (Callable) – Maps an array to another array

    • +

  • filter_fn (Callable, optional) – Filter to select which arrays to +map (default: Module.valid_parameter_filter()).

    • +
+
+
+
+ +
+
+apply_to_modules(apply_fn: Callable[[str, Module], Any])
+

Apply a function to all the modules in this instance (including this +instance).

+
+
Parameters:
+

apply_fn (Callable) – The function to apply to the modules.

+
+
+
+ +
+
+children()
+

Return the direct descendants of this Module instance.

+
+ +
+
+filter_and_map(filter_fn: Callable[[Module, str, Any], bool], map_fn: Optional[Callable] = None, is_leaf_fn: Optional[Callable[[Module, str, Any], bool]] = None)
+

Recursively filter the contents of the module using filter_fn, +namely only select keys and values where filter_fn returns true.

+

This is used to implement parameters() and trainable_parameters() +but it can also be used to extract any subset of the module’s parameters.

+
+
Parameters:
+
    +

  • filter_fn (Callable) – Given a value, the key in which it is found +and the containing module, decide whether to keep the value or +drop it.

    • +

  • map_fn (Callable, optional) – Optionally transform the value before +returning it.

    • +

  • is_leaf_fn (Callable, optional) – Given a value, the key in which it +is found and the containing module decide if it is a leaf.

    • +
+
+
Returns:
+

A dictionary containing the contents of the module recursively filtered

+
+
+
+ +
+
+freeze(*, recurse: bool = True, keys: Optional[Union[str, List[str]]] = None, strict: bool = False)
+

Freeze the Module’s parameters or some of them. Freezing a parameter means not +computing gradients for it.

+

This function is idempotent ie freezing a frozen model is a noop.

+

For instance to only train the attention parameters from a transformer:

+
+

model = … +model.freeze() +model.apply_to_modules(lambda k, v: v.unfreeze() if k.endswith(“attention”) else None)

+
+
+
Parameters:
+
    +

  • recurse (bool, optional) – If True then freeze the parameters of the +submodules as well (default: True).

    • +

  • keys (str or list[str], optional) – If provided then only these +parameters will be frozen otherwise all the parameters of a +module. For instance freeze all biases by calling +module.freeze(keys="bias").

    • +

  • strict (bool, optional) – If set to True validate that the passed keys exist +(default: False).

    • +
+
+
+
+ +
+
+leaf_modules()
+

Return the submodules that do not contain other modules.

+
+ +
+
+load_weights(file: str)
+

Load and update the model’s weights from a .npz file.

+
+ +
+
+modules()
+

Return a list with all the modules in this instance.

+
+
Returns:
+

A list of mlx.nn.Module instances.

+
+
+
+ +
+
+named_modules()
+

Return a list with all the modules in this instance and their name +with dot notation.

+
+
Returns:
+

A list of tuples (str, mlx.nn.Module).

+
+
+
+ +
+
+parameters()
+

Recursively return all the mlx.core.array members of this Module +as a dict of dicts and lists.

+
+ +
+
+save_weights(file: str)
+

Save the model’s weights to a .npz file.

+
+ +
+
+trainable_parameters()
+

Recursively return all the non frozen mlx.core.array members of +this Module as a dict of dicts and lists.

+
+ +
+
+unfreeze(*, recurse: bool = True, keys: Optional[Union[str, List[str]]] = None, strict: bool = False)
+

Unfreeze the Module’s parameters or some of them.

+

This function is idempotent ie unfreezing a model that is not frozen is +a noop.

+

For instance to only train the biases one can do:

+
+

model = … +model.freeze() +model.unfreeze(keys=”bias”)

+
+
+
Parameters:
+
    +

  • recurse (bool, optional) – If True then unfreeze the parameters of the +submodules as well (default: True).

    • +

  • keys (str or list[str], optional) – If provided then only these +parameters will be unfrozen otherwise all the parameters of a +module. For instance unfreeze all biases by calling +module.unfreeze(keys="bias").

    • +

  • strict (bool, optional) – If set to True validate that the passed keys exist +(default: False).

    • +
+
+
+
+ +
+
+update(parameters: dict)
+

Replace the parameters of this Module with the provided ones in the +dict of dicts and lists.

+

Commonly used by the optimizer to change the model to the updated +(optimized) parameters. Also used by the mlx.nn.value_and_grad() to set the +tracers in the model in order to compute gradients.

+

The passed in parameters dictionary need not be a full dictionary +similar to parameters(). Only the provided locations will be +updated.

+
+
Parameters:
+

parameters (dict) – A complete or partial dictionary of the modules +parameters.

+
+
+
+ +
+ +
+ + +
+
+
+ +
+ +
+

© Copyright 2023, MLX Contributors.

+
+ + Built with Sphinx using a + theme + provided by Read the Docs. + + +
+
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/ops.html b/docs/build/html/python/ops.html new file mode 100644 index 000000000..ea88a09ca --- /dev/null +++ b/docs/build/html/python/ops.html @@ -0,0 +1,485 @@ + + + + + + + Operations — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Operations

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

abs(a, /, *[, stream])

Element-wise absolute value.

add(a, b, /, *[, stream])

Element-wise addition.

all(a, /[, axis, keepdims, stream])

An and reduction over the given axes.

allclose(a, b, /[, rtol, atol, stream])

Approximate comparison of two arrays.

any(a, /[, axis, keepdims, stream])

An or reduction over the given axes.

arange(start, stop, step[, dtype, stream])

Generates ranges of numbers.

arccos(a, /, *[, stream])

Element-wise inverse cosine.

arccosh(a, /, *[, stream])

Element-wise inverse hyperbolic cosine.

arcsin(a, /, *[, stream])

Element-wise inverse sine.

arcsinh(a, /, *[, stream])

Element-wise inverse hyperbolic sine.

arctan(a, /, *[, stream])

Element-wise inverse tangent.

arctanh(a, /, *[, stream])

Element-wise inverse hyperbolic tangent.

argmax(a, /[, axis, keepdims, stream])

Indices of the maximum values along the axis.

argmin(a, /[, axis, keepdims, stream])

Indices of the minimum values along the axis.

argpartition(a, /, kth[, axis, stream])

Returns the indices that partition the array.

argsort(a, /[, axis, stream])

Returns the indices that sort the array.

array_equal(a, b, /, *[, equal_nan, stream])

Array equality check.

broadcast_to(a, /, shape, *[, stream])

Broadcast an array to the given shape.

concatenate(arrays, /[, axis, stream])

Concatenate the arrays along the given axis.

convolve(a, v, /[, mode, stream])

The discrete convolution of 1D arrays.

conv1d(input, weight, /[, stride, padding, ...])

1D convolution over an input with several channels

conv2d(input, weight, /[, stride, padding, ...])

2D convolution over an input with several channels

cos(a, /, *[, stream])

Element-wise cosine.

cosh(a, /, *[, stream])

Element-wise hyperbolic cosine.

divide(a, b, /, *[, stream])

Element-wise division.

equal(a, b, /, *[, stream])

Element-wise equality.

erf(a, /, *[, stream])

Element-wise error function.

erfinv(a, /, *[, stream])

Element-wise inverse of erf().

exp(a, /, *[, stream])

Element-wise exponential.

expand_dims(a, /, axis, *[, stream])

Add a size one dimension at the given axis.

full(shape, vals[, dtype, stream])

Construct an array with the given value.

greater(a, b, /, *[, stream])

Element-wise greater than.

greater_equal(a, b, /, *[, stream])

Element-wise greater or equal.

less(a, b, /, *[, stream])

Element-wise less than.

less_equal(a, b, /, *[, stream])

Element-wise less than or equal.

load(file, /, *[, stream])

Load array(s) from a binary file in .npy or .npz format.

log(a, /, *[, stream])

Element-wise natural logarithm.

log2(a, /, *[, stream])

Element-wise base-2 logarithm.

log10(a, /, *[, stream])

Element-wise base-10 logarithm.

log1p(a, /, *[, stream])

Element-wise natural log of one plus the array.

logaddexp(a, b, /, *[, stream])

Element-wise log-add-exp.

logical_not(a, /, *[, stream])

Element-wise logical not.

logsumexp(a, /[, axis, keepdims, stream])

A log-sum-exp reduction over the given axes.

matmul(a, b, /, *[, stream])

Matrix multiplication.

max(a, /[, axis, keepdims, stream])

An max reduction over the given axes.

maximum(a, b, /, *[, stream])

Element-wise maximum.

mean(a, /[, axis, keepdims, stream])

Compute the mean(s) over the given axes.

min(a, /[, axis, keepdims, stream])

An min reduction over the given axes.

minimum(a, b, /[, stream])

Element-wise minimum.

multiply(a, b, /, *[, stream])

Element-wise multiplication.

negative(a, /, *[, stream])

Element-wise negation.

ones(shape[, dtype, stream])

Construct an array of ones.

ones_like(a, /, *[, stream])

An array of ones like the input.

partition(a, /, kth[, axis, stream])

Returns a partitioned copy of the array such that the smaller kth elements are first.

pad(a, /, pad_width[, constant_values, stream])

Pad an array with a constant value

prod(a, /[, axis, keepdims, stream])

An product reduction over the given axes.

reciprocal(a, /, *[, stream])

Element-wise reciprocal.

reshape(a, /, shape, *[, stream])

Reshape an array while preserving the size.

rsqrt(a, /, *[, stream])

Element-wise reciprocal and square root.

save(file, arr, /[, retain_graph])

Save the array to a binary file in .npy format.

savez(file, /, *args, **kwargs)

Save several arrays to a binary file in uncompressed .npz format.

savez_compressed(file, /, *args, **kwargs)

Save several arrays to a binary file in compressed .npz format.

sigmoid(a, /, *[, stream])

Element-wise logistic sigmoid.

sign(a, /, *[, stream])

Element-wise sign.

sin(a, /, *[, stream])

Element-wise sine.

sinh(a, /, *[, stream])

Element-wise hyperbolic sine.

softmax(a, /[, axis, stream])

Perform the softmax along the given axis.

sort(a, /[, axis, stream])

Returns a sorted copy of the array.

split(a, /, indices_or_sections[, axis, stream])

Split an array along a given axis.

sqrt(a, /, *[, stream])

Element-wise square root.

square(a, /, *[, stream])

Element-wise square.

squeeze(a, /[, axis, stream])

Remove length one axes from an array.

stop_gradient(a, /, *[, stream])

Stop gradients from being computed.

subtract(a, b, /, *[, stream])

Element-wise subtraction.

sum(array, /[, axis, keepdims, stream])

Sum reduce the array over the given axes.

take(a, /, indices[, axis, stream])

Take elements along an axis.

take_along_axis(a, /, indices, axis, *[, stream])

Take values along an axis at the specified indices.

tan(a, /, *[, stream])

Element-wise tangent.

tanh(a, /, *[, stream])

Element-wise hyperbolic tangent.

transpose(a, /[, axes, stream])

Transpose the dimensions of the array.

var(a, /[, axis, keepdims, ddof, stream])

Compute the variance(s) over the given axes.

where(condition, x, y, /, *[, stream])

Select from x or y according to condition.

zeros(shape[, dtype, stream])

Construct an array of zeros.

zeros_like(a, /, *[, stream])

An array of zeros like the input.

+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/optimizers.html b/docs/build/html/python/optimizers.html new file mode 100644 index 000000000..fcbfedca6 --- /dev/null +++ b/docs/build/html/python/optimizers.html @@ -0,0 +1,189 @@ + + + + + + + Optimizers — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Optimizers

+

The optimizers in MLX can be used both with mlx.nn but also with pure +mlx.core functions. A typical example involves calling +Optimizer.update() to update a model’s parameters based on the loss +gradients and subsequently calling mlx.core.eval() to evaluate both the +model’s parameters and the optimizer state.

+
# Create a model
+model = MLP(num_layers, train_images.shape[-1], hidden_dim, num_classes)
+mx.eval(model.parameters())
+
+# Create the gradient function and the optimizer
+loss_and_grad_fn = nn.value_and_grad(model, loss_fn)
+optimizer = optim.SGD(learning_rate=learning_rate)
+
+for e in range(num_epochs):
+    for X, y in batch_iterate(batch_size, train_images, train_labels):
+        loss, grads = loss_and_grad_fn(model, X, y)
+
+        # Update the model with the gradients. So far no computation has happened.
+        optimizer.update(model, grads)
+
+        # Compute the new parameters but also the optimizer state.
+        mx.eval(model.parameters(), optimizer.state)
+
+
+ + + + + + + + + + + + + + + +

OptimizerState

The optimizer state implements a recursively defined collections.defaultdict, namely a missing key in an optimizer state is an OptimizerState.

Optimizer()

The base class for all optimizers.

SGD(learning_rate[, momentum])

Stochastic gradient descent optimizer.

Adam(learning_rate[, betas, eps])

Implementation of the Adam optimizer [1].

+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/random.html b/docs/build/html/python/random.html new file mode 100644 index 000000000..da3663da0 --- /dev/null +++ b/docs/build/html/python/random.html @@ -0,0 +1,207 @@ + + + + + + + Random — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Random

+

Random sampling functions in MLX use an implicit global PRNG state by default. +However, all function take an optional key keyword argument for when more +fine-grained control or explicit state management is needed.

+

For example, you can generate random numbers with:

+
for _ in range(3):
+  print(mx.random.uniform())
+
+
+

which will print a sequence of unique pseudo random numbers. Alternatively you +can explicitly set the key:

+
key = mx.random.key(0)
+for _ in range(3):
+  print(mx.random.uniform(key=key))
+
+
+

which will yield the same pseudo random number at each iteration.

+

Following JAX’s PRNG design +we use a splittable version of Threefry, which is a counter-based PRNG.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

seed(seed)

Seed the global PRNG.

key(seed)

Get a PRNG key from a seed.

split(key[, num, stream])

Split a PRNG key into sub keys.

bernoulli([p, shape, key, stream])

Generate Bernoulli random values.

categorical(logits[, axis, shape, ...])

Sample from a categorical distribution.

gumbel([shape, dtype, stream, key])

Sample from the standard Gumbel distribution.

normal([shape, dtype, key, stream])

Generate normally distributed random numbers.

randint(low, high[, shape, dtype, key, stream])

Generate random integers from the given interval.

uniform([low, high, shape, dtype, key, stream])

Generate uniformly distributed random numbers.

truncated_normal(lower, upper[, shape, ...])

Generate values from a truncated normal distribution.

+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/transforms.html b/docs/build/html/python/transforms.html new file mode 100644 index 000000000..24751e20d --- /dev/null +++ b/docs/build/html/python/transforms.html @@ -0,0 +1,173 @@ + + + + + + + Transforms — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Transforms

+ + + + + + + + + + + + + + + + + + + + + +

eval(*args[, retain_graph])

Evaluate an array or tree of array.

grad(fun[, argnums])

Returns a function which computes the gradient of fun.

value_and_grad(fun[, argnums])

Returns a function which computes the value and gradient of fun.

jvp(fun, primals, tangents)

Compute the Jacobian-vector product.

vjp(fun, primals, cotangents)

Compute the vector-Jacobian product.

vmap(fun[, in_axes, out_axes])

Returns a vectorized version of fun.

+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/python/tree_utils.html b/docs/build/html/python/tree_utils.html new file mode 100644 index 000000000..fe8cd09de --- /dev/null +++ b/docs/build/html/python/tree_utils.html @@ -0,0 +1,169 @@ + + + + + + + Tree Utils — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Tree Utils

+

In MLX we consider a python tree to be an arbitrarily nested collection of +dictionaries, lists and tuples without cycles. Functions in this module that +return python trees will be using the default python dict, list and +tuple but they can usually process objects that inherit from any of these.

+
+

Note

+

Dictionaries should have keys that are valid python identifiers.

+
+ + + + + + + + + + + + +

tree_flatten(tree[, prefix, is_leaf])

Flattens a python tree to a list of key, value tuples.

tree_unflatten(tree)

Recreate a python tree from its flat representation.

tree_map(fn, tree, *rest)

Applies fn to the leaves of the python tree tree and returns a new collection with the results.

+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/quick_start.html b/docs/build/html/quick_start.html new file mode 100644 index 000000000..cf9783c45 --- /dev/null +++ b/docs/build/html/quick_start.html @@ -0,0 +1,224 @@ + + + + + + + Quick Start Guide — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Quick Start Guide

+

MLX is a NumPy-like array framework designed for efficient and flexible +machine learning on Apple silicon. The Python API closely follows NumPy with +a few exceptions. MLX also has a fully featured C++ API which closely follows +the Python API.

+

The main differences between MLX and NumPy are:

+
+
    +

  • Composable function transformations: MLX has composable function +transformations for automatic differentiation, automatic vectorization, +and computation graph optimization.

    • +

  • Lazy computation: Computations in MLX are lazy. Arrays are only +materialized when needed.

    • +

  • Multi-device: Operations can run on any of the suppoorted devices (CPU, +GPU, …)

    • +
+
+

The design of MLX is strongly inspired by frameworks like PyTorch, Jax, and +ArrayFire. A noteable difference from these +frameworks and MLX is the unified memory model. Arrays in MLX live in shared +memory. Operations on MLX arrays can be performed on any of the supported +device types without performing data copies. Currently supported device types +are the CPU and GPU.

+
+

Basics

+

Import mlx.core and make an array:

+
>> import mlx.core as mx
+>> a = mx.array([1, 2, 3, 4])
+>> a.shape
+[4]
+>> a.dtype
+int32
+>> b = mx.array([1.0, 2.0, 3.0, 4.0])
+>> b.dtype
+float32
+
+
+

Operations in MLX are lazy. The outputs of MLX operations are not computed +until they are needed. To force an array to be evaluated use +eval(). Arrays will automatically be evaluated in a few cases. For +example, inspecting a scalar with array.item(), printing an array, +or converting an array from array to numpy.ndarray all +automatically evaluate the array.

+
>> c = a + b    # c not yet evaluated
+>> mx.eval(c)  # evaluates c
+>> c = a + b
+>> print(c)     # Also evaluates c
+array([2, 4, 6, 8], dtype=float32)
+>> c = a + b
+>> import numpy as np
+>> np.array(c)   # Also evaluates c
+array([2., 4., 6., 8.], dtype=float32)
+
+
+
+
+

Function and Graph Transformations

+

MLX has standard function transformations like grad() and vmap(). +Transformations can be composed arbitrarily. For example +grad(vmap(grad(fn))) (or any other composition) is allowed.

+
>> x = mx.array(0.0)
+>> mx.sin(x)
+array(0, dtype=float32)
+>> mx.grad(mx.sin)(x)
+array(1, dtype=float32)
+>> mx.grad(mx.grad(mx.sin))(x)
+array(-0, dtype=float32)
+
+
+

Other gradient transformations include vjp() for vector-Jacobian products +and jvp() for Jacobian-vector products.

+

Use value_and_grad() to efficiently compute both a function’s output and +gradient with respect to the function’s input.

+
+
+

Devices and Streams

+
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/docs/build/html/search.html b/docs/build/html/search.html new file mode 100644 index 000000000..f3b568f1f --- /dev/null +++ b/docs/build/html/search.html @@ -0,0 +1,154 @@ + + + + + + Search — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+
    +
    • +
  • Search
    • +
  • +
    • +
+
+
+
+
+ + + + +
+ +
+ +
+
+
+ +
+ +
+

© Copyright 2023, MLX Contributors.

+
+ + Built with Sphinx using a + theme + provided by Read the Docs. + + +
+
+
+
+
+ + + + + + + + + \ No newline at end of file diff --git a/docs/build/html/searchindex.js b/docs/build/html/searchindex.js new file mode 100644 index 000000000..170edaa63 --- /dev/null +++ b/docs/build/html/searchindex.js @@ -0,0 +1 @@ +Search.setIndex({"docnames": ["cpp/ops", "dev/extensions", "examples/linear_regression", "examples/llama-inference", "examples/mlp", "index", "install", "python/_autosummary/mlx.core.Device", "python/_autosummary/mlx.core.Dtype", "python/_autosummary/mlx.core.Stream", "python/_autosummary/mlx.core.abs", "python/_autosummary/mlx.core.add", "python/_autosummary/mlx.core.all", "python/_autosummary/mlx.core.allclose", "python/_autosummary/mlx.core.any", "python/_autosummary/mlx.core.arange", "python/_autosummary/mlx.core.arccos", "python/_autosummary/mlx.core.arccosh", "python/_autosummary/mlx.core.arcsin", "python/_autosummary/mlx.core.arcsinh", "python/_autosummary/mlx.core.arctan", "python/_autosummary/mlx.core.arctanh", "python/_autosummary/mlx.core.argmax", "python/_autosummary/mlx.core.argmin", "python/_autosummary/mlx.core.argpartition", "python/_autosummary/mlx.core.argsort", "python/_autosummary/mlx.core.array", "python/_autosummary/mlx.core.array.T", "python/_autosummary/mlx.core.array.abs", "python/_autosummary/mlx.core.array.all", "python/_autosummary/mlx.core.array.any", "python/_autosummary/mlx.core.array.argmax", "python/_autosummary/mlx.core.array.argmin", "python/_autosummary/mlx.core.array.astype", "python/_autosummary/mlx.core.array.cos", "python/_autosummary/mlx.core.array.dtype", "python/_autosummary/mlx.core.array.exp", "python/_autosummary/mlx.core.array.item", "python/_autosummary/mlx.core.array.log", "python/_autosummary/mlx.core.array.log1p", "python/_autosummary/mlx.core.array.logsumexp", "python/_autosummary/mlx.core.array.max", "python/_autosummary/mlx.core.array.mean", "python/_autosummary/mlx.core.array.min", "python/_autosummary/mlx.core.array.ndim", "python/_autosummary/mlx.core.array.prod", "python/_autosummary/mlx.core.array.reciprocal", "python/_autosummary/mlx.core.array.reshape", "python/_autosummary/mlx.core.array.rsqrt", "python/_autosummary/mlx.core.array.shape", "python/_autosummary/mlx.core.array.sin", "python/_autosummary/mlx.core.array.size", "python/_autosummary/mlx.core.array.split", "python/_autosummary/mlx.core.array.sqrt", "python/_autosummary/mlx.core.array.square", "python/_autosummary/mlx.core.array.sum", "python/_autosummary/mlx.core.array.tolist", "python/_autosummary/mlx.core.array.transpose", "python/_autosummary/mlx.core.array.var", "python/_autosummary/mlx.core.array_equal", "python/_autosummary/mlx.core.broadcast_to", "python/_autosummary/mlx.core.concatenate", "python/_autosummary/mlx.core.conv1d", "python/_autosummary/mlx.core.conv2d", "python/_autosummary/mlx.core.convolve", "python/_autosummary/mlx.core.cos", "python/_autosummary/mlx.core.cosh", "python/_autosummary/mlx.core.default_device", "python/_autosummary/mlx.core.default_stream", "python/_autosummary/mlx.core.divide", "python/_autosummary/mlx.core.equal", "python/_autosummary/mlx.core.erf", "python/_autosummary/mlx.core.erfinv", "python/_autosummary/mlx.core.eval", "python/_autosummary/mlx.core.exp", "python/_autosummary/mlx.core.expand_dims", "python/_autosummary/mlx.core.fft.fft", "python/_autosummary/mlx.core.fft.fft2", "python/_autosummary/mlx.core.fft.fftn", "python/_autosummary/mlx.core.fft.ifft", "python/_autosummary/mlx.core.fft.ifft2", "python/_autosummary/mlx.core.fft.ifftn", "python/_autosummary/mlx.core.fft.irfft", "python/_autosummary/mlx.core.fft.irfft2", "python/_autosummary/mlx.core.fft.irfftn", "python/_autosummary/mlx.core.fft.rfft", "python/_autosummary/mlx.core.fft.rfft2", "python/_autosummary/mlx.core.fft.rfftn", "python/_autosummary/mlx.core.full", "python/_autosummary/mlx.core.grad", "python/_autosummary/mlx.core.greater", "python/_autosummary/mlx.core.greater_equal", "python/_autosummary/mlx.core.jvp", "python/_autosummary/mlx.core.less", "python/_autosummary/mlx.core.less_equal", "python/_autosummary/mlx.core.load", "python/_autosummary/mlx.core.log", "python/_autosummary/mlx.core.log10", "python/_autosummary/mlx.core.log1p", "python/_autosummary/mlx.core.log2", "python/_autosummary/mlx.core.logaddexp", "python/_autosummary/mlx.core.logical_not", "python/_autosummary/mlx.core.logsumexp", "python/_autosummary/mlx.core.matmul", "python/_autosummary/mlx.core.max", "python/_autosummary/mlx.core.maximum", "python/_autosummary/mlx.core.mean", "python/_autosummary/mlx.core.min", "python/_autosummary/mlx.core.minimum", "python/_autosummary/mlx.core.multiply", "python/_autosummary/mlx.core.negative", "python/_autosummary/mlx.core.new_stream", "python/_autosummary/mlx.core.ones", "python/_autosummary/mlx.core.ones_like", "python/_autosummary/mlx.core.pad", "python/_autosummary/mlx.core.partition", "python/_autosummary/mlx.core.prod", "python/_autosummary/mlx.core.random.bernoulli", "python/_autosummary/mlx.core.random.categorical", "python/_autosummary/mlx.core.random.gumbel", "python/_autosummary/mlx.core.random.key", "python/_autosummary/mlx.core.random.normal", "python/_autosummary/mlx.core.random.randint", "python/_autosummary/mlx.core.random.seed", "python/_autosummary/mlx.core.random.split", "python/_autosummary/mlx.core.random.truncated_normal", "python/_autosummary/mlx.core.random.uniform", "python/_autosummary/mlx.core.reciprocal", "python/_autosummary/mlx.core.reshape", "python/_autosummary/mlx.core.rsqrt", "python/_autosummary/mlx.core.save", "python/_autosummary/mlx.core.savez", "python/_autosummary/mlx.core.savez_compressed", "python/_autosummary/mlx.core.set_default_device", "python/_autosummary/mlx.core.set_default_stream", "python/_autosummary/mlx.core.sigmoid", "python/_autosummary/mlx.core.sign", "python/_autosummary/mlx.core.sin", "python/_autosummary/mlx.core.sinh", "python/_autosummary/mlx.core.softmax", "python/_autosummary/mlx.core.sort", "python/_autosummary/mlx.core.split", "python/_autosummary/mlx.core.sqrt", "python/_autosummary/mlx.core.square", "python/_autosummary/mlx.core.squeeze", "python/_autosummary/mlx.core.stop_gradient", "python/_autosummary/mlx.core.subtract", "python/_autosummary/mlx.core.sum", "python/_autosummary/mlx.core.take", "python/_autosummary/mlx.core.take_along_axis", "python/_autosummary/mlx.core.tan", "python/_autosummary/mlx.core.tanh", "python/_autosummary/mlx.core.transpose", "python/_autosummary/mlx.core.value_and_grad", "python/_autosummary/mlx.core.var", "python/_autosummary/mlx.core.vjp", "python/_autosummary/mlx.core.vmap", "python/_autosummary/mlx.core.where", "python/_autosummary/mlx.core.zeros", "python/_autosummary/mlx.core.zeros_like", "python/_autosummary/mlx.nn.Conv1d", "python/_autosummary/mlx.nn.Conv2d", "python/_autosummary/mlx.nn.Embedding", "python/_autosummary/mlx.nn.GELU", "python/_autosummary/mlx.nn.GroupNorm", "python/_autosummary/mlx.nn.LayerNorm", "python/_autosummary/mlx.nn.Linear", "python/_autosummary/mlx.nn.MultiHeadAttention", "python/_autosummary/mlx.nn.RMSNorm", "python/_autosummary/mlx.nn.ReLU", "python/_autosummary/mlx.nn.RoPE", "python/_autosummary/mlx.nn.Sequential", "python/_autosummary/mlx.nn.SiLU", "python/_autosummary/mlx.nn.value_and_grad", "python/_autosummary/mlx.optimizers.Adam", "python/_autosummary/mlx.optimizers.Optimizer", "python/_autosummary/mlx.optimizers.OptimizerState", "python/_autosummary/mlx.optimizers.SGD", "python/_autosummary/mlx.utils.tree_flatten", "python/_autosummary/mlx.utils.tree_map", "python/_autosummary/mlx.utils.tree_unflatten", "python/_autosummary_functions/mlx.nn.gelu", "python/_autosummary_functions/mlx.nn.gelu_approx", "python/_autosummary_functions/mlx.nn.gelu_fast_approx", "python/_autosummary_functions/mlx.nn.relu", "python/_autosummary_functions/mlx.nn.silu", "python/array", "python/data_types", "python/devices_and_streams", "python/fft", "python/nn", "python/nn/module", "python/ops", "python/optimizers", "python/random", "python/transforms", "python/tree_utils", "quick_start", "using_streams"], "filenames": ["cpp/ops.rst", "dev/extensions.rst", "examples/linear_regression.rst", "examples/llama-inference.rst", "examples/mlp.rst", "index.rst", "install.rst", "python/_autosummary/mlx.core.Device.rst", "python/_autosummary/mlx.core.Dtype.rst", "python/_autosummary/mlx.core.Stream.rst", "python/_autosummary/mlx.core.abs.rst", "python/_autosummary/mlx.core.add.rst", "python/_autosummary/mlx.core.all.rst", "python/_autosummary/mlx.core.allclose.rst", "python/_autosummary/mlx.core.any.rst", "python/_autosummary/mlx.core.arange.rst", "python/_autosummary/mlx.core.arccos.rst", "python/_autosummary/mlx.core.arccosh.rst", "python/_autosummary/mlx.core.arcsin.rst", "python/_autosummary/mlx.core.arcsinh.rst", "python/_autosummary/mlx.core.arctan.rst", "python/_autosummary/mlx.core.arctanh.rst", "python/_autosummary/mlx.core.argmax.rst", "python/_autosummary/mlx.core.argmin.rst", "python/_autosummary/mlx.core.argpartition.rst", "python/_autosummary/mlx.core.argsort.rst", "python/_autosummary/mlx.core.array.rst", "python/_autosummary/mlx.core.array.T.rst", "python/_autosummary/mlx.core.array.abs.rst", "python/_autosummary/mlx.core.array.all.rst", "python/_autosummary/mlx.core.array.any.rst", "python/_autosummary/mlx.core.array.argmax.rst", "python/_autosummary/mlx.core.array.argmin.rst", "python/_autosummary/mlx.core.array.astype.rst", "python/_autosummary/mlx.core.array.cos.rst", "python/_autosummary/mlx.core.array.dtype.rst", "python/_autosummary/mlx.core.array.exp.rst", "python/_autosummary/mlx.core.array.item.rst", "python/_autosummary/mlx.core.array.log.rst", "python/_autosummary/mlx.core.array.log1p.rst", "python/_autosummary/mlx.core.array.logsumexp.rst", "python/_autosummary/mlx.core.array.max.rst", "python/_autosummary/mlx.core.array.mean.rst", "python/_autosummary/mlx.core.array.min.rst", "python/_autosummary/mlx.core.array.ndim.rst", "python/_autosummary/mlx.core.array.prod.rst", "python/_autosummary/mlx.core.array.reciprocal.rst", "python/_autosummary/mlx.core.array.reshape.rst", "python/_autosummary/mlx.core.array.rsqrt.rst", "python/_autosummary/mlx.core.array.shape.rst", "python/_autosummary/mlx.core.array.sin.rst", "python/_autosummary/mlx.core.array.size.rst", "python/_autosummary/mlx.core.array.split.rst", "python/_autosummary/mlx.core.array.sqrt.rst", "python/_autosummary/mlx.core.array.square.rst", "python/_autosummary/mlx.core.array.sum.rst", "python/_autosummary/mlx.core.array.tolist.rst", "python/_autosummary/mlx.core.array.transpose.rst", "python/_autosummary/mlx.core.array.var.rst", "python/_autosummary/mlx.core.array_equal.rst", "python/_autosummary/mlx.core.broadcast_to.rst", "python/_autosummary/mlx.core.concatenate.rst", "python/_autosummary/mlx.core.conv1d.rst", "python/_autosummary/mlx.core.conv2d.rst", "python/_autosummary/mlx.core.convolve.rst", "python/_autosummary/mlx.core.cos.rst", "python/_autosummary/mlx.core.cosh.rst", "python/_autosummary/mlx.core.default_device.rst", "python/_autosummary/mlx.core.default_stream.rst", "python/_autosummary/mlx.core.divide.rst", "python/_autosummary/mlx.core.equal.rst", "python/_autosummary/mlx.core.erf.rst", "python/_autosummary/mlx.core.erfinv.rst", "python/_autosummary/mlx.core.eval.rst", "python/_autosummary/mlx.core.exp.rst", "python/_autosummary/mlx.core.expand_dims.rst", "python/_autosummary/mlx.core.fft.fft.rst", "python/_autosummary/mlx.core.fft.fft2.rst", "python/_autosummary/mlx.core.fft.fftn.rst", "python/_autosummary/mlx.core.fft.ifft.rst", "python/_autosummary/mlx.core.fft.ifft2.rst", "python/_autosummary/mlx.core.fft.ifftn.rst", "python/_autosummary/mlx.core.fft.irfft.rst", "python/_autosummary/mlx.core.fft.irfft2.rst", "python/_autosummary/mlx.core.fft.irfftn.rst", "python/_autosummary/mlx.core.fft.rfft.rst", "python/_autosummary/mlx.core.fft.rfft2.rst", "python/_autosummary/mlx.core.fft.rfftn.rst", "python/_autosummary/mlx.core.full.rst", "python/_autosummary/mlx.core.grad.rst", "python/_autosummary/mlx.core.greater.rst", "python/_autosummary/mlx.core.greater_equal.rst", "python/_autosummary/mlx.core.jvp.rst", "python/_autosummary/mlx.core.less.rst", "python/_autosummary/mlx.core.less_equal.rst", "python/_autosummary/mlx.core.load.rst", "python/_autosummary/mlx.core.log.rst", "python/_autosummary/mlx.core.log10.rst", "python/_autosummary/mlx.core.log1p.rst", "python/_autosummary/mlx.core.log2.rst", "python/_autosummary/mlx.core.logaddexp.rst", "python/_autosummary/mlx.core.logical_not.rst", "python/_autosummary/mlx.core.logsumexp.rst", "python/_autosummary/mlx.core.matmul.rst", "python/_autosummary/mlx.core.max.rst", "python/_autosummary/mlx.core.maximum.rst", "python/_autosummary/mlx.core.mean.rst", "python/_autosummary/mlx.core.min.rst", "python/_autosummary/mlx.core.minimum.rst", "python/_autosummary/mlx.core.multiply.rst", "python/_autosummary/mlx.core.negative.rst", "python/_autosummary/mlx.core.new_stream.rst", "python/_autosummary/mlx.core.ones.rst", "python/_autosummary/mlx.core.ones_like.rst", "python/_autosummary/mlx.core.pad.rst", "python/_autosummary/mlx.core.partition.rst", "python/_autosummary/mlx.core.prod.rst", "python/_autosummary/mlx.core.random.bernoulli.rst", "python/_autosummary/mlx.core.random.categorical.rst", "python/_autosummary/mlx.core.random.gumbel.rst", "python/_autosummary/mlx.core.random.key.rst", "python/_autosummary/mlx.core.random.normal.rst", "python/_autosummary/mlx.core.random.randint.rst", "python/_autosummary/mlx.core.random.seed.rst", "python/_autosummary/mlx.core.random.split.rst", "python/_autosummary/mlx.core.random.truncated_normal.rst", "python/_autosummary/mlx.core.random.uniform.rst", "python/_autosummary/mlx.core.reciprocal.rst", "python/_autosummary/mlx.core.reshape.rst", "python/_autosummary/mlx.core.rsqrt.rst", "python/_autosummary/mlx.core.save.rst", "python/_autosummary/mlx.core.savez.rst", "python/_autosummary/mlx.core.savez_compressed.rst", "python/_autosummary/mlx.core.set_default_device.rst", "python/_autosummary/mlx.core.set_default_stream.rst", "python/_autosummary/mlx.core.sigmoid.rst", "python/_autosummary/mlx.core.sign.rst", "python/_autosummary/mlx.core.sin.rst", "python/_autosummary/mlx.core.sinh.rst", "python/_autosummary/mlx.core.softmax.rst", "python/_autosummary/mlx.core.sort.rst", "python/_autosummary/mlx.core.split.rst", "python/_autosummary/mlx.core.sqrt.rst", "python/_autosummary/mlx.core.square.rst", "python/_autosummary/mlx.core.squeeze.rst", "python/_autosummary/mlx.core.stop_gradient.rst", "python/_autosummary/mlx.core.subtract.rst", "python/_autosummary/mlx.core.sum.rst", "python/_autosummary/mlx.core.take.rst", "python/_autosummary/mlx.core.take_along_axis.rst", "python/_autosummary/mlx.core.tan.rst", "python/_autosummary/mlx.core.tanh.rst", "python/_autosummary/mlx.core.transpose.rst", "python/_autosummary/mlx.core.value_and_grad.rst", "python/_autosummary/mlx.core.var.rst", "python/_autosummary/mlx.core.vjp.rst", "python/_autosummary/mlx.core.vmap.rst", "python/_autosummary/mlx.core.where.rst", "python/_autosummary/mlx.core.zeros.rst", "python/_autosummary/mlx.core.zeros_like.rst", "python/_autosummary/mlx.nn.Conv1d.rst", "python/_autosummary/mlx.nn.Conv2d.rst", "python/_autosummary/mlx.nn.Embedding.rst", "python/_autosummary/mlx.nn.GELU.rst", "python/_autosummary/mlx.nn.GroupNorm.rst", "python/_autosummary/mlx.nn.LayerNorm.rst", "python/_autosummary/mlx.nn.Linear.rst", "python/_autosummary/mlx.nn.MultiHeadAttention.rst", "python/_autosummary/mlx.nn.RMSNorm.rst", "python/_autosummary/mlx.nn.ReLU.rst", "python/_autosummary/mlx.nn.RoPE.rst", "python/_autosummary/mlx.nn.Sequential.rst", "python/_autosummary/mlx.nn.SiLU.rst", "python/_autosummary/mlx.nn.value_and_grad.rst", "python/_autosummary/mlx.optimizers.Adam.rst", "python/_autosummary/mlx.optimizers.Optimizer.rst", "python/_autosummary/mlx.optimizers.OptimizerState.rst", "python/_autosummary/mlx.optimizers.SGD.rst", "python/_autosummary/mlx.utils.tree_flatten.rst", "python/_autosummary/mlx.utils.tree_map.rst", "python/_autosummary/mlx.utils.tree_unflatten.rst", "python/_autosummary_functions/mlx.nn.gelu.rst", "python/_autosummary_functions/mlx.nn.gelu_approx.rst", "python/_autosummary_functions/mlx.nn.gelu_fast_approx.rst", "python/_autosummary_functions/mlx.nn.relu.rst", "python/_autosummary_functions/mlx.nn.silu.rst", "python/array.rst", "python/data_types.rst", "python/devices_and_streams.rst", "python/fft.rst", "python/nn.rst", "python/nn/module.rst", "python/ops.rst", "python/optimizers.rst", "python/random.rst", "python/transforms.rst", "python/tree_utils.rst", "quick_start.rst", "using_streams.rst"], "titles": ["Operations", "Developer Documentation", "Linear Regression", "LLM inference", "Multi-Layer Perceptron", "MLX", "Build and Install", "mlx.core.Device", "mlx.core.Dtype", "mlx.core.Stream", "mlx.core.abs", "mlx.core.add", "mlx.core.all", "mlx.core.allclose", "mlx.core.any", "mlx.core.arange", "mlx.core.arccos", "mlx.core.arccosh", "mlx.core.arcsin", "mlx.core.arcsinh", "mlx.core.arctan", "mlx.core.arctanh", "mlx.core.argmax", "mlx.core.argmin", "mlx.core.argpartition", "mlx.core.argsort", "mlx.core.array", "mlx.core.array.T", "mlx.core.array.abs", "mlx.core.array.all", "mlx.core.array.any", "mlx.core.array.argmax", "mlx.core.array.argmin", "mlx.core.array.astype", "mlx.core.array.cos", "mlx.core.array.dtype", "mlx.core.array.exp", "mlx.core.array.item", "mlx.core.array.log", "mlx.core.array.log1p", "mlx.core.array.logsumexp", "mlx.core.array.max", "mlx.core.array.mean", "mlx.core.array.min", "mlx.core.array.ndim", "mlx.core.array.prod", "mlx.core.array.reciprocal", "mlx.core.array.reshape", "mlx.core.array.rsqrt", "mlx.core.array.shape", "mlx.core.array.sin", "mlx.core.array.size", "mlx.core.array.split", "mlx.core.array.sqrt", "mlx.core.array.square", "mlx.core.array.sum", "mlx.core.array.tolist", "mlx.core.array.transpose", "mlx.core.array.var", "mlx.core.array_equal", "mlx.core.broadcast_to", "mlx.core.concatenate", "mlx.core.conv1d", "mlx.core.conv2d", "mlx.core.convolve", "mlx.core.cos", "mlx.core.cosh", "mlx.core.default_device", "mlx.core.default_stream", "mlx.core.divide", "mlx.core.equal", "mlx.core.erf", "mlx.core.erfinv", "mlx.core.eval", "mlx.core.exp", "mlx.core.expand_dims", "mlx.core.fft.fft", "mlx.core.fft.fft2", "mlx.core.fft.fftn", "mlx.core.fft.ifft", "mlx.core.fft.ifft2", "mlx.core.fft.ifftn", "mlx.core.fft.irfft", "mlx.core.fft.irfft2", "mlx.core.fft.irfftn", "mlx.core.fft.rfft", "mlx.core.fft.rfft2", "mlx.core.fft.rfftn", "mlx.core.full", "mlx.core.grad", "mlx.core.greater", "mlx.core.greater_equal", "mlx.core.jvp", "mlx.core.less", "mlx.core.less_equal", "mlx.core.load", "mlx.core.log", "mlx.core.log10", "mlx.core.log1p", "mlx.core.log2", "mlx.core.logaddexp", "mlx.core.logical_not", "mlx.core.logsumexp", "mlx.core.matmul", "mlx.core.max", "mlx.core.maximum", "mlx.core.mean", "mlx.core.min", "mlx.core.minimum", "mlx.core.multiply", "mlx.core.negative", "mlx.core.new_stream", "mlx.core.ones", "mlx.core.ones_like", "mlx.core.pad", "mlx.core.partition", "mlx.core.prod", "mlx.core.random.bernoulli", "mlx.core.random.categorical", "mlx.core.random.gumbel", "mlx.core.random.key", "mlx.core.random.normal", "mlx.core.random.randint", "mlx.core.random.seed", "mlx.core.random.split", "mlx.core.random.truncated_normal", "mlx.core.random.uniform", "mlx.core.reciprocal", "mlx.core.reshape", "mlx.core.rsqrt", "mlx.core.save", "mlx.core.savez", "mlx.core.savez_compressed", "mlx.core.set_default_device", "mlx.core.set_default_stream", "mlx.core.sigmoid", "mlx.core.sign", "mlx.core.sin", "mlx.core.sinh", "mlx.core.softmax", "mlx.core.sort", "mlx.core.split", "mlx.core.sqrt", "mlx.core.square", "mlx.core.squeeze", "mlx.core.stop_gradient", "mlx.core.subtract", "mlx.core.sum", "mlx.core.take", "mlx.core.take_along_axis", "mlx.core.tan", "mlx.core.tanh", "mlx.core.transpose", "mlx.core.value_and_grad", "mlx.core.var", "mlx.core.vjp", "mlx.core.vmap", "mlx.core.where", "mlx.core.zeros", "mlx.core.zeros_like", "mlx.nn.Conv1d", "mlx.nn.Conv2d", "mlx.nn.Embedding", "mlx.nn.GELU", "mlx.nn.GroupNorm", "mlx.nn.LayerNorm", "mlx.nn.Linear", "mlx.nn.MultiHeadAttention", "mlx.nn.RMSNorm", "mlx.nn.ReLU", "mlx.nn.RoPE", "mlx.nn.Sequential", "mlx.nn.SiLU", "mlx.nn.value_and_grad", "mlx.optimizers.Adam", "mlx.optimizers.Optimizer", "mlx.optimizers.OptimizerState", "mlx.optimizers.SGD", "mlx.utils.tree_flatten", "mlx.utils.tree_map", "mlx.utils.tree_unflatten", "mlx.nn.gelu", "mlx.nn.gelu_approx", "mlx.nn.gelu_fast_approx", "mlx.nn.relu", "mlx.nn.silu", "Array", "Data Types", "Devices and Streams", "FFT", "Neural Networks", "mlx.nn.Module", "Operations", "Optimizers", "Random", "Transforms", "Tree Utils", "Quick Start Guide", "Using Streams"], "terms": {"mlx": [1, 2, 3, 4, 6, 190, 193, 194, 196, 197], "provid": [1, 3, 167, 179, 190, 191, 198], "open": [1, 15, 122, 126], "flexibl": [1, 197], "which": [1, 3, 4, 15, 33, 73, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 89, 92, 95, 118, 119, 128, 130, 131, 132, 144, 148, 153, 155, 156, 163, 170, 191, 194, 197, 198], "user": [1, 3, 190], "mai": 1, "add": [1, 3, 75, 100, 114, 160, 161], "special": 1, "without": [1, 3, 145, 167, 190, 196, 197], "much": [1, 3], "hassl": 1, "while": [1, 3, 128, 170], "librari": [1, 6, 190], "suppli": 1, "effici": [1, 3, 170, 197], "can": [1, 3, 6, 11, 15, 47, 57, 69, 70, 73, 90, 91, 93, 94, 100, 105, 108, 109, 117, 118, 122, 125, 126, 146, 153, 162, 171, 190, 191, 193, 194, 196, 197, 198], "compos": [1, 190, 197], "ani": [1, 3, 15, 163, 178, 179, 180, 190, 191, 196, 197], "number": [1, 15, 51, 63, 89, 92, 114, 118, 121, 124, 126, 153, 155, 156, 160, 161, 164, 167, 194, 198], "applic": 1, "aris": 1, "case": [1, 3, 78, 81, 82, 84, 85, 86, 87, 103, 128, 144, 197, 198], "where": [1, 4, 153, 156, 160, 161, 163, 164, 165, 168, 172, 181, 182, 183, 185, 191], "new": [1, 4, 60, 128, 152, 167, 179, 191, 193], "function": [1, 2, 3, 4, 13, 26, 71, 72, 73, 89, 92, 103, 135, 153, 155, 156, 163, 171, 173, 179, 181, 182, 183, 190, 191, 193, 194, 196], "highli": 1, "optim": [1, 2, 4, 5, 191, 197], "ar": [1, 2, 3, 4, 13, 15, 59, 60, 64, 77, 78, 80, 81, 83, 84, 86, 87, 92, 103, 114, 115, 117, 118, 119, 122, 125, 126, 131, 132, 144, 148, 155, 156, 160, 161, 164, 165, 167, 178, 179, 190, 191, 196, 197], "need": [1, 3, 4, 59, 190, 191, 194, 197], "For": [1, 3, 180, 191, 194, 197], "you": [1, 3, 6, 194], "design": [1, 2, 194, 197], "your": [1, 3, 6, 191], "own": [1, 6], "link": [1, 6], "top": 1, "core": [1, 2, 3, 4, 190, 191, 193, 197], "we": [1, 2, 3, 4, 162, 171, 190, 194, 196], "inner": 1, "work": [1, 3], "go": [1, 3], "over": [1, 3, 4, 12, 14, 22, 23, 24, 25, 62, 63, 78, 81, 84, 87, 102, 104, 106, 107, 115, 116, 129, 139, 140, 147, 154, 160, 161, 164, 165, 168], "simpl": [1, 3, 4, 6, 162, 190], "learn": [1, 2, 4, 164, 165, 168, 177, 197], "step": [1, 3, 4, 15], "involv": [1, 193], "ad": [1, 2, 191], "let": [1, 2, 3], "s": [1, 2, 3, 4, 6, 35, 44, 77, 78, 80, 81, 83, 84, 86, 87, 89, 95, 106, 118, 153, 154, 156, 173, 175, 190, 191, 193, 194, 197], "sai": [1, 3], "would": [1, 3], "like": [1, 3, 113, 159, 197], "an": [1, 3, 4, 8, 12, 14, 26, 60, 62, 63, 73, 88, 104, 107, 112, 113, 114, 116, 128, 141, 144, 148, 149, 156, 158, 159, 164, 165, 166, 167, 175, 176, 178, 182, 190, 191, 194, 196, 197, 198], "take": [1, 3, 4, 89, 92, 105, 108, 113, 149, 153, 155, 156, 159, 194, 198], "two": [1, 11, 13, 59, 69, 70, 77, 80, 86, 90, 91, 93, 94, 100, 103, 105, 108, 109], "arrai": [1, 3, 4, 5, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 59, 60, 61, 62, 63, 64, 65, 66, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 124, 125, 126, 127, 128, 129, 130, 131, 132, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 190, 191, 197], "x": [1, 2, 3, 4, 71, 119, 131, 135, 157, 163, 164, 165, 168, 169, 172, 179, 181, 182, 183, 184, 185, 190, 191, 193, 197], "y": [1, 2, 3, 4, 157, 164, 165, 168, 190, 193], "scale": [1, 3, 167], "them": [1, 3, 190, 191], "both": [1, 11, 69, 70, 90, 91, 93, 94, 100, 105, 108, 109, 118, 146, 193, 197], "some": [1, 2, 3, 4, 191], "coeffic": 1, "alpha": 1, "beta": [1, 164, 165, 174], "respect": [1, 2, 4, 89, 153, 163, 164, 165, 179, 190, 191, 197], "togeth": [1, 4, 179], "get": [1, 2, 4, 63, 120, 176], "z": 1, "well": [1, 3, 167, 190, 191], "veri": [1, 3, 167], "easili": 1, "do": [1, 3, 6, 191], "just": [1, 4], "write": [1, 3, 190], "out": 1, "follow": [1, 3, 4, 6, 15, 64, 174, 177, 182, 183, 194, 197], "import": [1, 2, 3, 4, 131, 153, 178, 179, 180, 190, 191, 197], "mx": [1, 2, 3, 4, 131, 153, 169, 184, 190, 191, 193, 194, 197, 198], "def": [1, 2, 3, 4, 153, 190, 191], "simple_axpbi": 1, "float": [1, 11, 13, 15, 26, 56, 59, 60, 69, 70, 88, 90, 91, 93, 94, 100, 101, 105, 108, 109, 114, 117, 122, 125, 126, 146, 157, 164, 165, 168, 174, 177, 187, 191], "return": [1, 2, 3, 4, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 33, 37, 49, 56, 59, 60, 61, 62, 63, 64, 65, 66, 69, 70, 71, 72, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 124, 125, 126, 127, 128, 129, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 173, 178, 179, 180, 190, 191, 196], "thi": [1, 3, 4, 12, 13, 14, 15, 22, 23, 24, 25, 73, 89, 92, 100, 102, 103, 104, 106, 107, 115, 116, 118, 139, 140, 141, 147, 148, 153, 154, 182, 183, 190, 191, 196], "perform": [1, 3, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 103, 139, 148, 164, 190, 197], "leav": [1, 179], "differenti": [1, 197], "howev": [1, 163, 164, 190, 194], "vector": [1, 2, 92, 148, 155, 156, 162, 197], "math": [1, 3], "often": 1, "realiz": 1, "axpbi": 1, "routin": 1, "defin": [1, 2, 3, 4, 6, 176, 178], "same": [1, 3, 6, 59, 60, 63, 64, 82, 85, 86, 87, 89, 92, 114, 118, 155, 157, 164, 190, 191, 194], "realli": 1, "part": 1, "doe": [1, 3, 190], "fast": [1, 163, 183], "so": [1, 3, 193], "decid": [1, 191], "want": [1, 3], "reli": 1, "acceler": 1, "framework": [1, 197], "continu": 1, "impos": 1, "our": [1, 3, 171, 174], "assumpt": 1, "also": [1, 3, 4, 11, 69, 70, 78, 81, 84, 87, 90, 91, 93, 94, 100, 105, 108, 109, 146, 167, 173, 176, 181, 190, 191, 193, 197, 198], "assum": [1, 3, 164, 179, 190], "how": [1, 3, 4, 160, 161, 162, 167, 190], "gradient": [1, 2, 4, 89, 145, 153, 173, 177, 190, 191, 193, 197], "ins": 1, "what": [1, 3], "coincid": 1, "right": [1, 163, 182, 183], "place": [1, 3], "cours": 1, "The": [1, 3, 4, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 33, 35, 44, 49, 56, 59, 60, 61, 62, 63, 64, 65, 66, 69, 70, 71, 72, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 124, 125, 126, 127, 128, 135, 136, 137, 138, 139, 140, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 164, 165, 166, 167, 168, 170, 171, 173, 175, 176, 177, 178, 179, 180, 187, 191, 193, 197, 198], "structur": [1, 73], "from": [1, 3, 4, 83, 84, 86, 87, 88, 95, 103, 113, 117, 118, 119, 120, 122, 125, 131, 144, 145, 146, 148, 149, 157, 159, 167, 178, 179, 180, 190, 191, 196, 197], "frontend": 1, "api": [1, 197], "redirect": 1, "when": [1, 3, 160, 161, 190, 191, 194, 197], "appropri": 1, "fallback": 1, "metal": [1, 6], "vjp": [1, 197], "jvp": [1, 197], "In": [1, 3, 4, 103, 164, 174, 179, 190, 191, 196], "one": [1, 3, 56, 63, 75, 98, 103, 118, 144, 146, 191], "sentenc": 1, "comput": [1, 2, 3, 4, 6, 89, 92, 100, 106, 139, 145, 153, 154, 155, 164, 165, 168, 173, 182, 183, 190, 191, 193, 197], "graph": [1, 3, 4, 73, 130], "rule": 1, "evalu": [1, 3, 4, 73, 92, 130, 155, 190, 191, 193, 197], "said": [1, 3], "start": [1, 2, 3, 5, 6, 15, 141], "discuss": 1, "more": [1, 4, 8, 56, 103, 190, 194], "detail": [1, 8, 174, 190], "thei": [1, 2, 3, 64, 171, 191, 196, 197], "c": [1, 3, 160, 161, 187, 197], "scalar": [1, 11, 13, 37, 56, 59, 69, 70, 89, 90, 91, 93, 94, 100, 101, 103, 105, 108, 109, 114, 122, 125, 126, 146, 153, 173, 197], "sum": [1, 2, 11, 102, 139], "elementwis": 1, "numpi": [1, 3, 4, 11, 13, 15, 26, 60, 69, 70, 90, 91, 93, 94, 100, 103, 105, 108, 109, 146, 197], "style": [1, 11, 13, 69, 70, 90, 91, 93, 94, 100, 103, 105, 108, 109, 146], "broadcast": [1, 11, 13, 60, 69, 70, 88, 90, 91, 93, 94, 100, 103, 105, 108, 109, 117, 118, 125, 126, 146, 149, 157, 167], "between": [1, 197], "input": [1, 2, 3, 10, 11, 12, 13, 14, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 59, 60, 61, 62, 63, 64, 65, 66, 69, 70, 71, 72, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 89, 90, 91, 92, 93, 94, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 113, 114, 115, 116, 124, 127, 128, 129, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 156, 157, 159, 160, 161, 162, 164, 165, 166, 167, 168, 170, 197], "upcast": 1, "const": 1, "factor": 1, "streamordevic": 1, "stream": [1, 5, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 28, 29, 30, 31, 32, 33, 34, 36, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 50, 52, 53, 54, 55, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 68, 69, 70, 71, 72, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 90, 91, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 121, 122, 124, 125, 126, 127, 128, 129, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 154, 157, 158, 159], "schedul": 1, "itself": 1, "call": [1, 3, 4, 27, 162, 171, 190, 191, 193], "other": [1, 3, 167, 190, 191, 197], "within": [1, 24], "simplest": 1, "wai": [1, 3, 190], "about": [1, 3, 4], "term": 1, "exist": [1, 3, 191], "auto": 1, "ax": [1, 12, 14, 22, 23, 57, 75, 77, 78, 80, 81, 83, 84, 86, 87, 102, 104, 106, 107, 114, 116, 139, 144, 147, 152, 154], "multipli": 1, "earlier": 1, "goal": 1, "themselv": 1, "contain": [1, 3, 49, 73, 82, 83, 84, 101, 141, 157, 190, 191], "act": 1, "data": [1, 4, 8, 15, 85, 86, 88, 112, 125, 158, 197], "nor": 1, "rather": 1, "easi": [1, 190], "interfac": 1, "block": [1, 3], "A": [1, 3, 6, 49, 59, 89, 92, 102, 103, 117, 118, 119, 121, 122, 125, 126, 141, 153, 155, 156, 164, 165, 168, 171, 173, 174, 178, 179, 180, 183, 190, 191, 193, 197], "It": [1, 3, 175, 190], "creat": [1, 3, 6, 190, 191, 193], "output": [1, 3, 12, 13, 14, 15, 22, 23, 24, 60, 82, 85, 86, 87, 88, 89, 102, 104, 106, 107, 112, 113, 115, 116, 117, 118, 119, 121, 122, 125, 126, 131, 132, 139, 144, 147, 149, 153, 154, 155, 156, 157, 158, 159, 160, 161, 166, 167, 197], "given": [1, 12, 14, 24, 60, 61, 73, 75, 76, 77, 78, 79, 80, 81, 85, 86, 87, 88, 102, 104, 106, 107, 116, 122, 139, 141, 147, 154, 167, 191], "set": [1, 3, 4, 163, 166, 170, 176, 191, 194], "further": 1, "class": [1, 3, 4, 7, 8, 9, 26, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 174, 175, 176, 177, 181, 182, 183, 184, 185, 191], "under": 1, "These": [1, 149], "word": 1, "bit": [1, 187, 191], "abstract": 1, "back": [1, 3], "give": [1, 3, 4, 24], "ourselv": 1, "concret": 1, "imag": [1, 161], "public": [1, 190], "explicit": [1, 194], "alpha_": 1, "beta_": 1, "must": [1, 6, 73, 88, 117, 118, 122, 125, 126, 157], "know": [1, 3], "popul": 1, "To": [1, 2, 3, 6, 197], "avoid": 1, "unecessari": 1, "alloc": [1, 191], "respons": 1, "space": 1, "void": 1, "eval_cpu": 1, "std": 1, "overrid": 1, "eval_gpu": 1, "jacobian": [1, 92, 155, 197], "product": [1, 92, 103, 116, 155, 167, 197], "primal": [1, 92, 155], "tangent": [1, 20, 21, 92, 150, 151], "int": [1, 3, 4, 7, 9, 11, 12, 14, 15, 22, 23, 24, 25, 26, 29, 30, 31, 32, 40, 41, 42, 43, 45, 49, 52, 55, 56, 58, 59, 60, 61, 62, 63, 69, 70, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 93, 94, 100, 101, 102, 104, 105, 106, 107, 108, 109, 112, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 128, 139, 140, 141, 144, 146, 147, 148, 149, 152, 153, 154, 156, 157, 158, 160, 161, 162, 164, 165, 166, 167, 168, 170, 190, 191], "argnum": [1, 89, 153], "cotan": 1, "accross": [1, 164], "pair": [1, 114, 170], "repres": [1, 3], "axi": [1, 3, 4, 12, 14, 22, 23, 24, 25, 29, 30, 31, 32, 40, 41, 42, 43, 45, 52, 55, 58, 61, 75, 76, 79, 82, 83, 84, 85, 86, 87, 102, 104, 106, 107, 114, 115, 116, 118, 139, 140, 141, 144, 147, 148, 149, 152, 154, 156], "correspond": [1, 12, 14, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 102, 104, 107, 116, 147, 156, 179], "dimens": [1, 3, 12, 14, 22, 23, 44, 49, 56, 63, 75, 83, 84, 86, 87, 102, 103, 104, 106, 107, 116, 118, 124, 147, 149, 152, 154, 160, 161, 164, 165, 167, 168, 170], "vmap": [1, 197], "print": [1, 2, 3, 4, 178, 179, 180, 190, 194, 197], "ostream": 1, "os": 1, "equival": [1, 27, 47, 57, 163], "check": [1, 59], "bool": [1, 11, 12, 14, 22, 23, 26, 29, 30, 31, 32, 40, 41, 42, 43, 45, 55, 56, 58, 59, 60, 69, 70, 73, 88, 90, 91, 93, 94, 100, 101, 102, 104, 105, 106, 107, 108, 109, 114, 116, 117, 122, 125, 126, 130, 146, 147, 154, 157, 160, 161, 164, 165, 166, 170, 191], "is_equival": 1, "privat": 1, "fall": 1, "eval": [1, 2, 3, 4, 190, 191, 193, 197], "deriv": 1, "base": [1, 73, 97, 99, 175, 191, 193, 194], "abov": [1, 3], "demonstr": 1, "treat": [1, 59, 83, 84, 86, 87, 148], "paramet": [1, 2, 3, 4, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 33, 59, 60, 61, 62, 63, 64, 65, 66, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 170, 171, 173, 175, 177, 178, 179, 180, 191, 193], "produc": [1, 167], "through": [1, 145], "construct": [1, 4, 88, 112, 158], "its": [1, 6, 103, 115, 124, 173, 180, 190], "type": [1, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 33, 49, 56, 59, 60, 61, 62, 63, 64, 65, 66, 69, 70, 71, 72, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 124, 125, 126, 127, 128, 129, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 175, 178, 190, 197], "shape": [1, 3, 4, 47, 59, 60, 62, 63, 76, 79, 82, 85, 86, 87, 88, 92, 103, 112, 113, 117, 118, 119, 121, 122, 125, 126, 128, 149, 155, 157, 158, 159, 160, 161, 190, 193, 197], "pass": [1, 3, 4, 47, 57, 114, 153, 171, 173, 178, 190, 191], "re": [1, 4], "now": [1, 3], "promot": 1, "dtype": [1, 3, 15, 26, 33, 56, 88, 112, 119, 121, 122, 125, 126, 158, 187, 197], "promoted_dtyp": 1, "promote_typ": 1, "float32": [1, 15, 112, 119, 121, 125, 126, 158, 187, 197], "non": [1, 191], "point": [1, 2, 3, 6, 187], "out_dtyp": 1, "is_floating_point": 1, "cast": [1, 33, 85, 86, 87, 191], "up": [1, 3], "determin": 1, "x_cast": 1, "astyp": [1, 3, 191], "y_cast": 1, "broadcasted_input": 1, "broadcast_arrai": 1, "out_shap": 1, "0": [1, 2, 3, 4, 6, 7, 15, 52, 58, 61, 62, 63, 89, 114, 117, 126, 141, 153, 154, 156, 160, 161, 163, 164, 165, 169, 174, 177, 178, 182, 183, 184, 190, 191, 194, 197], "unique_ptr": 1, "make_uniqu": 1, "to_stream": 1, "handl": [1, 190], "resolv": 1, "No": [1, 3], "happen": [1, 3, 193], "alon": 1, "effect": 1, "onli": [1, 3, 59, 62, 63, 187, 190, 191, 197], "execut": [1, 6], "depend": [1, 2, 56], "devic": [1, 5, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 28, 29, 30, 31, 32, 33, 34, 36, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 50, 52, 53, 54, 55, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 90, 91, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 121, 122, 124, 125, 126, 127, 128, 129, 133, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 154, 157, 158, 159, 198], "specifi": [1, 15, 33, 63, 83, 84, 88, 89, 112, 118, 148, 149, 152, 153, 156, 158], "memori": [1, 191, 197], "ha": [1, 3, 4, 56, 82, 83, 85, 86, 87, 89, 118, 191, 193, 197], "been": [1, 3], "try": 1, "naiv": 1, "gener": [1, 2, 15, 83, 84, 117, 121, 122, 125, 126, 194, 198], "version": [1, 6, 100, 102, 139, 156, 194], "declar": 1, "member": [1, 190, 191], "method": [1, 3, 7, 8, 9, 26, 174, 175, 176, 177], "each": [1, 49, 73, 103, 114, 118, 131, 132, 141, 152, 156, 157, 162, 164, 194], "element": [1, 10, 11, 16, 17, 18, 19, 20, 21, 24, 65, 66, 69, 70, 71, 72, 74, 90, 91, 93, 94, 96, 97, 98, 99, 100, 101, 105, 108, 109, 110, 115, 127, 129, 135, 136, 137, 138, 142, 143, 146, 148, 150, 151, 153, 157, 170, 172, 185], "find": [1, 2], "pointwis": 1, "captur": [1, 190], "templat": 1, "axpby_impl": 1, "typenam": 1, "t": [1, 3, 71, 153, 174, 177, 190], "readi": 1, "fill": [1, 88, 113, 159], "malloc_or_wait": 1, "synchron": 1, "avail": [1, 2, 3, 4, 6, 8, 187], "There": [1, 190], "wait": [1, 3], "here": [1, 3], "request": 1, "pressur": 1, "condit": [1, 157], "set_data": 1, "nbyte": 1, "collect": [1, 176, 179, 196], "pointer": 1, "x_ptr": 1, "y_ptr": 1, "out_ptr": 1, "relev": 1, "static_cast": 1, "size_t": 1, "out_idx": 1, "size": [1, 3, 4, 49, 63, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 118, 128, 141, 144, 160, 161, 162], "map": [1, 4, 95, 162, 179, 191], "linear": [1, 3, 4, 5, 163, 169, 172, 179, 181, 182, 183, 184, 185, 190, 191], "indic": [1, 13, 22, 23, 24, 25, 73, 89, 141, 148, 149, 153], "offset": [1, 3], "x_offset": 1, "elem_to_loc": 1, "stride": [1, 62, 63, 160, 161, 170], "y_offset": 1, "contigu": 1, "regularli": 1, "default": [1, 6, 12, 14, 15, 22, 23, 24, 25, 59, 61, 62, 63, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 89, 102, 104, 106, 107, 112, 115, 116, 117, 118, 119, 121, 122, 124, 125, 126, 128, 130, 140, 141, 144, 147, 152, 153, 154, 156, 158, 160, 161, 167, 170, 176, 187, 191, 194, 196, 198], "row": 1, "major": 1, "henc": 1, "doesn": [1, 190], "additon": 1, "abl": 1, "all": [1, 4, 6, 13, 24, 63, 73, 78, 81, 84, 87, 103, 114, 115, 144, 167, 175, 190, 191, 194, 197, 198], "incom": 1, "accordingli": 1, "dispatch": 1, "float16": [1, 187, 191], "bfloat16": 1, "complex64": 1, "throw": 1, "error": [1, 71, 72, 141, 163, 181, 182, 183], "encount": 1, "unexpect": [1, 15], "regist": [1, 4], "op": 1, "contruct": 1, "assert": 1, "2": [1, 2, 3, 4, 63, 71, 77, 80, 82, 83, 84, 85, 86, 87, 99, 103, 124, 161, 163, 168, 174, 182, 187, 190, 191, 197], "1": [1, 3, 4, 15, 24, 25, 62, 63, 76, 77, 79, 80, 82, 83, 84, 85, 86, 87, 103, 115, 118, 126, 135, 140, 148, 153, 160, 161, 163, 164, 165, 168, 170, 174, 177, 182, 183, 187, 191, 193, 197], "correct": [1, 174], "els": [1, 3, 190, 191], "float16_t": 1, "bfloat16_t": 1, "complex64_t": 1, "runtime_error": 1, "support": [1, 3, 6, 13, 62, 63, 103, 197], "have": [1, 3, 6, 59, 83, 84, 86, 87, 103, 118, 167, 171, 178, 196], "rememb": 1, "3": [1, 3, 6, 194, 197], "complic": 1, "keep": [1, 12, 14, 22, 23, 102, 104, 106, 107, 116, 147, 154, 190, 191], "mind": [1, 3], "half": [1, 15, 122, 126, 170], "precis": [1, 3, 163, 190], "direct": [1, 3, 191], "fix": [1, 3], "possibl": [1, 3, 103, 141, 162], "due": 1, "transpos": [1, 3, 27], "aren": 1, "guarante": 1, "fit": 1, "requir": [1, 3, 190], "column": 1, "inplac": 1, "expect": [1, 3, 160, 161, 167], "answer": 1, "copi": [1, 3, 115, 140, 197], "simpli": [1, 3, 6, 169, 184, 191], "catlas_saxpbi": 1, "axpby_impl_acceler": 1, "first": [1, 2, 3, 4, 6, 89, 103, 115, 124, 153, 164, 174, 178, 190], "mode": [1, 64], "i": [1, 3, 6, 92, 160, 161, 190], "e": [1, 4, 6, 71, 92, 135, 160, 161, 164, 165, 168, 190, 193, 198], "match": [1, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87], "transposit": 1, "data_s": 1, "items": 1, "flag": 1, "copy_inplac": 1, "copytyp": 1, "n": [1, 3, 26, 62, 63, 76, 78, 79, 81, 82, 85, 87, 154, 160, 161], "incx": 1, "inci": 1, "great": 1, "But": 1, "criteria": 1, "luckili": 1, "alwai": [1, 178], "With": 1, "final": [1, 2, 3, 4], "singl": [1, 4, 73, 92, 114, 155], "row_contigu": 1, "col_contigu": 1, "common": 1, "hit": 1, "mileston": 1, "enough": 1, "run": [1, 3, 4, 6, 191, 197, 198], "If": [1, 3, 12, 14, 15, 22, 23, 24, 25, 56, 59, 61, 64, 73, 85, 86, 87, 88, 89, 102, 103, 104, 106, 107, 112, 114, 115, 116, 118, 139, 140, 141, 147, 148, 149, 153, 154, 156, 158, 160, 161, 164, 165, 166, 167, 170, 171, 179, 191, 198], "plan": 1, "stop": [1, 3, 15, 145], "enjoi": 1, "speed": 1, "appl": [1, 3, 6, 197], "silicon": [1, 3, 6, 197], "address": 1, "shade": 1, "languag": [1, 187], "kernel": [1, 62, 63], "written": 1, "help": [1, 3], "resourc": 1, "walkthrough": 1, "pipelin": 1, "specif": 1, "cpp": 1, "algorithm": 1, "launch": 1, "exactli": [1, 3], "mani": [1, 141, 160, 161, 162, 167], "thread": 1, "pick": 1, "updat": [1, 2, 3, 4, 177, 179, 191, 193], "assign": [1, 191], "axpby_gener": 1, "buffer": [1, 26], "constant": [1, 3, 6, 114, 164, 165, 168], "4": [1, 3, 131, 187, 197], "5": [1, 2, 3, 6, 117], "x_stride": 1, "6": [1, 3, 131, 182, 183, 197], "y_stride": 1, "7": [1, 3], "ndim": 1, "8": [1, 3, 187, 197], "uint": 1, "index": [1, 7, 9, 24, 75, 89, 115, 148, 149, 153], "thread_position_in_grid": 1, "convert": [1, 56, 197], "instanti": [1, 4], "uniqu": [1, 194], "host": 1, "name": [1, 95, 131, 132, 164, 176, 190, 191], "identifi": [1, 178, 196], "instantiate_axpbi": 1, "type_nam": 1, "host_nam": 1, "axpby_general_": 1, "bflot16": 1, "compil": [1, 6], "mlx_ext": 1, "metallib": [1, 6], "see": [1, 3, 4, 8, 28, 29, 30, 31, 32, 34, 36, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 50, 52, 53, 54, 55, 57, 58, 163, 181, 182, 183], "later": [1, 6], "co": 1, "locat": [1, 191], "share": [1, 197], "register_librari": 1, "potenti": 1, "path": [1, 6, 131, 132], "tri": 1, "load": [1, 4, 191], "hasn": 1, "alreadi": [1, 3], "static": [1, 6], "object": [1, 8, 26, 37, 56, 95, 130, 131, 132, 156, 178, 196], "why": [1, 3], "packag": [1, 2, 4], "process": [1, 3, 64, 162, 179, 196], "logic": [1, 101], "grid": 1, "shown": 1, "below": [1, 187], "prepar": [1, 3], "carri": 1, "should": [1, 2, 3, 4, 6, 73, 92, 149, 153, 155, 160, 161, 167, 171, 178, 190, 191, 196, 198], "d": [1, 3, 103, 148, 174, 180], "ostringstream": 1, "kname": 1, "axpby_": 1, "general_": 1, "type_to_nam": 1, "make": [1, 3, 4, 6, 103, 190, 197], "sure": [1, 3, 6, 190], "look": [1, 3], "folder": 1, "get_colocated_mtllib_path": 1, "get_kernel": 1, "str": [1, 64, 95, 130, 131, 132, 178, 180, 191], "encod": [1, 170], "compute_encod": 1, "get_command_encod": 1, "setcomputepipelinest": 1, "those": [1, 3, 190], "decelar": 1, "nelem": 1, "set_array_buff": 1, "setbyt": 1, "sizeof": 1, "threadgroup": 1, "higher": 1, "than": [1, 3, 56, 64, 90, 91, 93, 94, 103, 170, 179], "max": [1, 105], "allow": [1, 175, 190, 191, 197], "tgp_size": 1, "min": [1, 108], "maxtotalthreadsperthreadgroup": 1, "3d": 1, "mtl": 1, "group_dim": 1, "grid_dim": 1, "divd": 1, "among": 1, "dispatchthread": 1, "few": [1, 3, 4, 197], "thing": [1, 3], "note": [1, 3, 6, 13, 62, 63, 83, 84, 118, 190], "befor": [1, 3, 24, 115, 130, 191], "move": 1, "track": [1, 190], "activ": [1, 190], "command": 1, "instead": [1, 190], "end_encod": 1, "end": 1, "until": [1, 197], "limit": 1, "flush": 1, "enqueu": 1, "commit": 1, "associ": [1, 131, 132], "suggest": 1, "deeper": 1, "dive": 1, "studi": 1, "come": [1, 3], "far": [1, 193], "built": [1, 6], "includ": [1, 191, 197, 198], "forward": [1, 153], "diff": 1, "push": 1, "along": [1, 22, 23, 61, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 139, 141, 148, 149], "primtiv": 1, "similarli": [1, 6, 103], "scale_arr": 1, "contribut": 1, "tangent_x": 1, "tangent_i": 1, "revers": [1, 152], "arg": [1, 3, 8, 26, 47, 57, 73, 131, 132], "push_back": 1, "fulli": [1, 197], "primitv": 1, "overal": 1, "directori": [1, 3, 6], "extens": [1, 187], "h": [1, 62, 63, 161], "mlx_sample_extens": 1, "__init__": [1, 3, 4, 7, 8, 9, 26, 190, 191], "py": [1, 3], "cmakelist": 1, "txt": 1, "setup": [1, 2, 4], "strucutr": 1, "hold": [1, 3, 8, 175], "instal": 1, "pybind11": [1, 6], "sinc": [1, 3, 4, 191], "compon": [1, 3], "etc": [1, 190], "becom": 1, "pybind11_modul": 1, "m": 1, "doc": [1, 4], "sampl": [1, 2, 3, 117, 118, 119, 122, 125, 126, 194], "_a": 1, "pos_onli": 1, "kw_onli": 1, "none": [1, 3, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 28, 29, 30, 31, 32, 33, 34, 36, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 50, 52, 53, 54, 55, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 90, 91, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 112, 113, 114, 115, 116, 117, 118, 119, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 154, 156, 157, 158, 159, 163, 167, 178, 191], "r": [1, 3, 153], "pbdoc": 1, "most": [1, 118, 190], "complex": [1, 11, 26, 59, 60, 69, 70, 83, 84, 85, 86, 87, 88, 90, 91, 93, 94, 100, 101, 105, 108, 109, 114, 117, 122, 125, 126, 146, 157, 178, 190], "addit": [1, 3, 11, 164, 165, 167, 168, 191], "bell": 1, "whistl": 1, "liter": 1, "string": 1, "modul": [1, 3, 4, 171, 173, 196], "ensur": 1, "caster": 1, "find_packag": 1, "config": 1, "add_librari": 1, "sourc": [1, 152], "target_sourc": 1, "cmake_current_list_dir": 1, "header": 1, "target_include_directori": 1, "target_link_librari": 1, "attach": 1, "conveni": [1, 4], "mlx_build_metallib": 1, "target": [1, 153], "destin": 1, "automat": [1, 197], "practic": 1, "mlx_build_met": [1, 6], "mlx_ext_metallib": 1, "titl": 1, "include_dir": 1, "project_source_dir": 1, "mlx_include_dir": 1, "output_directori": 1, "cmake_library_output_directori": 1, "add_depend": 1, "endif": 1, "pybind11_add_modul": 1, "build_shared_lib": 1, "target_link_opt": 1, "wl": 1, "rpath": 1, "loader_path": 1, "onc": 1, "describ": 1, "util": [1, 3, 5, 131], "__name__": [1, 3], "__main__": [1, 3], "descript": [1, 3, 187], "ext_modul": 1, "cmakeextens": 1, "cmdclass": 1, "build_ext": 1, "cmakebuild": 1, "package_dir": 1, "package_data": 1, "dylib": 1, "zip_saf": 1, "fals": [1, 3, 12, 14, 22, 23, 29, 30, 31, 32, 40, 41, 42, 43, 45, 55, 58, 59, 73, 102, 104, 106, 107, 116, 147, 154, 157, 164, 166, 170, 178, 187, 191], "python_requir": 1, "even": [1, 3], "though": [1, 3], "j8": 1, "libmlx_ext": 1, "cpython": 1, "3x": 1, "darwin": 1, "pip": [1, 6], "after": [1, 3, 4, 24, 115, 164, 165, 167], "plai": [1, 3], "ones": [1, 3, 113, 131, 191], "b": [1, 3, 11, 13, 59, 69, 70, 90, 91, 93, 94, 100, 103, 105, 108, 109, 146, 153, 197], "f": [1, 2, 4, 190], "item": [1, 2, 3, 4, 179, 197], "true": [1, 2, 3, 59, 130, 139, 157, 160, 161, 164, 165, 166, 170, 178, 187, 191], "quick": [1, 5], "benchmark": 1, "compar": [1, 59], "time": [1, 3, 6, 190], "set_default_devic": 1, "256": [1, 4], "512": [1, 3], "random": [1, 2, 3, 4, 5, 198], "normal": [1, 2, 3, 125, 164, 165, 168, 176], "bench": 1, "warm": 1, "rang": [1, 2, 3, 4, 15, 182, 183, 193, 194], "100": [1, 2, 3], "5000": 1, "simple_tim": 1, "custom_tim": 1, "3f": [1, 4], "custom": 1, "114": 1, "109": 1, "modest": 1, "improv": [1, 3], "awai": [1, 3], "good": 1, "nn": [1, 3, 4, 131, 179, 190, 193], "grad": [1, 2, 4, 153, 193, 197], "simplifi": 1, "full": [1, 4, 47, 57, 64, 139, 191], "implement": [2, 4, 162, 167, 170, 171, 174, 175, 176, 191], "basic": 2, "model": [2, 4, 131, 167, 173, 179, 190, 191, 193, 197], "problem": [2, 4, 190], "metadata": 2, "num_featur": 2, "num_exampl": 2, "1_000": 2, "num_it": 2, "10_000": 2, "iter": [2, 4, 179, 194], "sgd": [2, 4, 193], "lr": 2, "01": 2, "rate": 2, "ll": [2, 4], "synthet": 2, "dataset": 2, "matrix": [2, 103], "ground": [2, 3], "truth": 2, "w_star": 2, "valu": [2, 3, 10, 15, 22, 23, 37, 56, 59, 73, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 114, 117, 118, 119, 121, 122, 125, 126, 148, 149, 153, 156, 167, 173, 176, 178, 179, 187, 191], "gaussian": [2, 163, 181, 182, 183], "nois": 2, "exampl": [2, 3, 4, 15, 148, 193, 194, 197], "noisi": 2, "label": 2, "ep": [2, 164, 165, 168, 174], "1e": [2, 4, 13, 164, 165, 168, 174], "us": [2, 3, 4, 5, 6, 15, 103, 128, 162, 163, 166, 167, 175, 178, 182, 183, 190, 191, 193, 194, 196, 197], "weight": [2, 62, 63, 179, 190, 191], "squar": [2, 3, 129, 142, 153, 168, 179, 190], "loss": [2, 4, 153, 190, 193], "loss_fn": [2, 4, 193], "w": [2, 63, 153, 161, 177], "mean": [2, 3, 4, 153, 164, 168, 190, 191], "grad_fn": 2, "initi": [2, 3, 164, 165, 168, 190, 191], "randomli": [2, 3], "Then": [2, 6], "repeatedli": 2, "_": [2, 3, 190, 194], "verifi": 2, "close": [2, 13, 197], "error_norm": 2, "5f": 2, "someth": [2, 3], "00005": 2, "00364": 2, "complet": [2, 3, 191], "logist": [2, 135, 172, 182, 183, 185], "github": [2, 4, 6], "repo": [2, 4, 6], "enabl": [3, 73], "larg": [3, 190], "ish": 3, "transform": [3, 5, 73, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 164, 165, 166, 173, 190, 191], "compromis": 3, "eas": 3, "llama": 3, "famili": 3, "less": [3, 24, 94, 115, 170], "200": 3, "line": 3, "python": [3, 37, 49, 56, 73, 178, 179, 180, 191, 196, 197], "neural": [3, 5, 162, 191], "network": [3, 5, 162, 191], "build": [3, 5, 191], "concis": 3, "architectur": 3, "notabl": 3, "rope": 3, "posit": [3, 24, 115, 160, 161, 167, 170, 179, 190], "option": [3, 12, 14, 15, 22, 23, 24, 25, 26, 31, 32, 61, 62, 63, 64, 73, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 102, 104, 106, 107, 112, 114, 115, 116, 117, 118, 119, 121, 122, 124, 125, 126, 128, 130, 139, 140, 141, 144, 147, 148, 149, 152, 153, 154, 156, 158, 160, 161, 167, 178, 191, 194, 198], "kei": [3, 117, 118, 119, 121, 122, 124, 125, 126, 167, 176, 178, 179, 191, 194, 196], "cach": 3, "concaten": 3, "project": [3, 167], "llamaattent": 3, "self": [3, 4, 7, 9, 26, 27, 28, 29, 30, 31, 32, 33, 34, 36, 37, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 50, 52, 53, 54, 55, 56, 57, 58, 190, 191], "dim": [3, 162, 164, 165, 167, 168, 170], "num_head": [3, 167], "super": [3, 4, 190, 191], "tradit": [3, 170], "query_proj": 3, "bia": [3, 160, 161, 166, 174, 179, 191], "key_proj": 3, "value_proj": 3, "out_proj": [3, 191], "__call__": [3, 4, 190, 191], "queri": [3, 167], "mask": [3, 167], "extract": [3, 190, 191], "l": [3, 4, 160, 190], "reshap": 3, "combin": 3, "key_cach": 3, "value_cach": 3, "sqrt": [3, 71, 164, 165, 168, 174], "score": 3, "softmax": 3, "values_hat": 3, "rm": 3, "swiglu": 3, "rmsnorm": 3, "llamaencoderlay": 3, "mlp_dim": 3, "norm1": 3, "norm2": 3, "linear1": 3, "linear2": 3, "linear3": 3, "sigmoid": [3, 172, 182, 183, 185], "instanc": [3, 171, 180, 190, 191], "embed": 3, "emb": [3, 162], "token": [3, 162], "num_lay": [3, 4, 193], "vocab_s": 3, "norm": [3, 164], "multiheadattent": 3, "create_additive_causal_mask": 3, "list": [3, 8, 12, 14, 26, 29, 30, 40, 41, 42, 43, 45, 49, 52, 55, 56, 58, 60, 61, 73, 75, 77, 78, 80, 81, 83, 84, 86, 87, 88, 89, 92, 102, 104, 106, 107, 112, 114, 116, 117, 118, 119, 121, 122, 125, 126, 128, 139, 141, 144, 147, 152, 153, 154, 155, 158, 174, 178, 180, 190, 191, 196], "still": 3, "consid": [3, 13, 59, 164, 178, 196], "train": [3, 4, 191], "ignor": 3, "whatsoev": 3, "rest": [3, 170, 179], "subsect": 3, "prompt": 3, "autoregress": 3, "yield": [3, 4, 194], "temp": 3, "causal": 3, "save": [3, 95, 131, 132, 191], "append": [3, 103], "store": 3, "per": [3, 4, 164, 165, 168, 175], "care": 3, "last": [3, 25, 56, 78, 81, 83, 84, 86, 87, 103, 118, 140, 160, 161, 164], "logit": [3, 118], "next": [3, 4], "categor": 3, "lazili": [3, 190], "noth": [3, 190], "yet": [3, 190, 191, 197], "forc": [3, 4, 190, 197], "choos": [3, 170], "pars": 3, "feed": 3, "loop": [3, 4], "unsqueez": 3, "sequenc": [3, 160, 194], "length": [3, 144, 160], "len": [3, 78, 81, 84, 87], "overwrit": 3, "discard": [3, 178], "old": 3, "moment": [3, 174], "anymor": 3, "everyth": 3, "small": [3, 164, 165, 168], "10": [3, 4, 97, 131, 179, 190], "12": 3, "8192": 3, "1024": 3, "actual": [3, 15, 191], "materi": [3, 197], "could": [3, 190], "20_000": 3, "machin": [3, 197], "8gb": 3, "ram": 3, "32": [3, 4, 187], "44": 3, "doubl": 3, "bracket": 3, "becaus": [3, 190], "batch": [3, 103, 160, 161, 167], "zip": [3, 4], "haven": 3, "anyth": [3, 153], "result": [3, 15, 56, 95, 103, 157, 179], "similar": [3, 167, 191], "runtim": 3, "section": [3, 141], "access": [3, 37, 190, 191], "origin": [3, 174], "sentencepiec": 3, "pytorch": [3, 164, 197], "compat": [3, 118], "npz": [3, 95, 131, 132, 191], "file": [3, 6, 95, 130, 131, 132, 191], "directli": 3, "argpars": 3, "itertool": [3, 179], "starmap": [3, 179], "np": [3, 4, 197], "torch": 3, "map_torch_to_mlx": 3, "tok_embed": 3, "elif": 3, "replac": [3, 191], "attention_norm": 3, "ffn_norm": 3, "wq": 3, "wk": 3, "wv": 3, "wo": 3, "w1": 3, "w2": 3, "w3": 3, "ffn": 3, "separ": [3, 47, 57, 164], "submodul": [3, 4, 190, 191], "feed_forward": 3, "parser": 3, "argumentpars": 3, "add_argu": 3, "torch_weight": 3, "output_fil": 3, "parse_arg": 3, "state": [3, 4, 175, 176, 190, 193, 194], "savez": 3, "k": [3, 191], "v": [3, 64, 191], "left": [3, 163, 170, 182, 183], "disk": 3, "text": 3, "format": [3, 95, 130, 131, 132], "oper": [3, 5, 33, 139, 145, 149, 190, 197, 198], "dictionari": [3, 175, 176, 178, 190, 191, 196], "represent": [3, 178, 180], "tree_unflatten": 3, "helper": 3, "weight_fil": 3, "incur": 3, "sever": [3, 62, 63, 131, 132], "unnecessari": 3, "futur": 3, "pth": 3, "current": [3, 6, 62, 63, 190, 197], "around": 3, "m1": 3, "ultra": 3, "7b": 3, "me": 3, "ishmael": 3, "year": 3, "ago": 3, "never": 3, "long": 3, "info": 3, "247": 3, "press": 3, "enter": 3, "littl": 3, "monei": 3, "my": 3, "purs": 3, "greater": [3, 24, 91, 115], "consequ": 3, "walk": 3, "down": 3, "gower": 3, "street": 3, "afternoon": 3, "heavi": 3, "rain": 3, "saw": 3, "off": [3, 6], "man": 3, "rag": 3, "who": 3, "sat": 3, "upon": [3, 179], "hi": 3, "bundl": 3, "hard": 3, "wet": 3, "he": 3, "were": 3, "cry": 3, "watch": 3, "him": 3, "observ": 3, "numer": [3, 100, 102, 139, 164, 165, 168], "crowd": 3, "wa": [3, 176], "hurri": 3, "437": 3, "330": 3, "second": [3, 103, 153, 174], "spent": 3, "amount": 3, "39": 3, "ms": 3, "By": 3, "bigger": 3, "remain": [3, 153], "almost": 3, "nobodi": 3, "took": 3, "least": 3, "notic": 3, "distanc": 3, "had": 3, "doubt": 3, "minut": 3, "straight": 3, "slowli": 3, "rais": [3, 141], "ey": 3, "speak": 3, "resum": 3, "postur": 3, "stood": 3, "feel": 3, "pain": 3, "heart": 3, "smile": 3, "face": 3, "am": 3, "someon": 3, "three": 3, "quarter": 3, "hour": 3, "made": 3, "immedi": [3, 191], "repli": 3, "again": [3, 190], "hand": 3, "did": 3, "accustom": 3, "thu": [3, 190], "question": 3, "reason": 3, "tell": 3, "understand": 3, "579": 3, "690": 3, "num": [3, 124], "500": 3, "628": 3, "went": 3, "nervou": 3, "trembl": 3, "told": 3, "And": 3, "perhap": 3, "surpris": 3, "matter": [3, 190], "shall": 3, "anyhow": 3, "friend": 3, "ye": 3, "slight": 3, "kind": 3, "longer": [3, 64], "soon": 3, "unless": [3, 191], "unlik": [3, 13], "strang": 3, "amus": 3, "That": 3, "secret": 3, "disappoint": 3, "mine": 3, "cannot": 3, "happi": 3, "ask": 3, "Is": 3, "shop": 3, "bui": 3, "food": 3, "633": 3, "21": 3, "475": 3, "su": 3, "j": [3, 6, 174], "lu": 3, "pan": 3, "murtadha": 3, "wen": 3, "liu": 3, "2021": 3, "roform": 3, "enhanc": 3, "rotari": [3, 170], "arxiv": [3, 164, 165, 168, 170], "preprint": 3, "2104": [3, 170], "09864": [3, 170], "zhang": 3, "sennrich": 3, "2019": 3, "root": [3, 129, 142, 168], "advanc": 3, "inform": [3, 4, 163, 167], "system": 3, "shazeer": 3, "2020": 3, "glu": 3, "variant": 3, "2002": 3, "05202": 3, "classifi": 4, "mnist": 4, "As": [4, 148], "mlp": [4, 190, 193], "inherit": [4, 196], "standard": [4, 37, 56, 103, 119, 197], "idiom": 4, "input_dim": [4, 166], "hidden_dim": [4, 191, 193], "output_dim": [4, 166], "layer_s": 4, "idim": 4, "odim": 4, "maximum": [4, 22, 169, 182, 183, 184, 190, 191], "cross": 4, "entropi": 4, "sub": [4, 124], "commonli": [4, 191], "cross_entropi": 4, "accuraci": 4, "valid": [4, 64, 156, 178, 191, 196], "eval_fn": 4, "argmax": 4, "num_class": [4, 193], "batch_siz": [4, 193], "num_epoch": [4, 193], "learning_r": [4, 174, 177, 193], "train_imag": [4, 193], "train_label": [4, 193], "test_imag": 4, "test_label": 4, "shuffl": 4, "minibatch": 4, "batch_iter": [4, 193], "perm": 4, "permut": 4, "id": 4, "put": 4, "trainabl": [4, 173, 190, 191], "loss_and_grad_fn": [4, 193], "value_and_grad": [4, 190, 191, 193, 197], "epoch": 4, "test": [4, 6], "confus": 4, "decent": 4, "95": 4, "guid": 5, "regress": 5, "multi": [5, 160, 161, 197], "layer": [5, 164, 165, 166, 171, 191], "perceptron": 5, "llm": 5, "infer": [5, 88], "develop": 5, "document": [5, 47, 57], "fft": 5, "tree": [5, 73, 89, 153, 156, 175, 178, 179, 180], "intern": 6, "repositori": 6, "http": [6, 164, 165, 168, 170], "com": 6, "17": 6, "g": [6, 177, 190, 198], "clang": 6, "cmake": 6, "24": 6, "clone": 6, "git": 6, "ml": 6, "explor": 6, "cd": 6, "brew": 6, "conda": 6, "global": [6, 123, 194], "env": 6, "cmake_build_parallel_level": 6, "mkdir": 6, "p": [6, 117, 174], "either": [6, 11, 47, 56, 57, 69, 70, 90, 91, 93, 94, 100, 103, 105, 108, 109, 146, 153, 171], "libmlx": 6, "preprocessor": 6, "metal_path": 6, "mlx_build_test": 6, "ON": 6, "mlx_build_exampl": 6, "mlx_build_benchmark": 6, "mlx_build_python_bind": 6, "devicetyp": 7, "attribut": [7, 8, 9, 26], "kwarg": [8, 26, 131, 132, 198], "union": [10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 28, 29, 30, 31, 32, 33, 34, 36, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 50, 52, 53, 54, 55, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 69, 70, 71, 72, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 112, 113, 114, 115, 116, 117, 118, 119, 121, 122, 124, 125, 126, 127, 128, 129, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 157, 158, 159, 161, 191], "wise": [10, 11, 16, 17, 18, 19, 20, 21, 65, 66, 69, 70, 71, 72, 74, 90, 91, 93, 94, 96, 97, 98, 99, 100, 101, 105, 108, 109, 110, 127, 129, 135, 136, 137, 138, 142, 143, 146, 150, 151, 172, 185], "absolut": [10, 13, 182, 183], "semant": [11, 60, 69, 70, 90, 91, 93, 94, 100, 103, 105, 108, 109, 146], "keepdim": [12, 14, 22, 23, 29, 30, 31, 32, 40, 41, 42, 43, 45, 55, 58, 102, 104, 106, 107, 116, 139, 147, 154], "reduct": [12, 14, 102, 104, 107, 116], "reduc": [12, 14, 22, 23, 102, 104, 106, 107, 116, 147, 154], "unspecifi": [12, 14, 15, 22, 23, 24, 25, 61, 88, 89, 102, 104, 106, 107, 112, 115, 116, 139, 140, 147, 148, 153, 154, 158, 198], "entir": [12, 14, 22, 23, 102, 104, 106, 107, 116, 147, 154], "singleton": [12, 14, 22, 23, 102, 103, 104, 106, 107, 116, 147, 154], "rtol": 13, "05": [13, 164, 165, 168], "atol": 13, "08": [13, 174], "approxim": [13, 163, 181, 182, 183], "comparison": [13, 70, 90, 91, 93, 94], "equal": [13, 24, 59, 91, 94, 115, 122, 141], "ab": [13, 153, 164, 165, 168, 170], "array_equ": 13, "rel": 13, "toler": 13, "boolean": [13, 59, 101, 187], "interv": [15, 122, 126], "increment": 15, "otherwis": [15, 178, 191], "int32": [15, 122, 187, 197], "convent": [15, 64], "lead": 15, "fraction": 15, "integr": [15, 148], "invers": [16, 17, 18, 19, 20, 21, 72, 79, 80, 81, 82, 83, 84], "cosin": [16, 17, 65, 66], "hyperbol": [17, 19, 21, 66, 138, 151], "sine": [18, 19, 137, 138], "minimum": [22, 23], "kth": [24, 115], "partit": 24, "order": [24, 115, 164, 171, 190, 191], "undefin": [24, 115], "sort": [24, 25, 115], "partiton": 24, "flatten": [24, 25, 115, 140, 148, 149, 178], "dimension": [26, 76, 77, 78, 79, 80, 81, 85, 86, 87, 160, 161, 162, 166], "overload": 26, "val": [26, 88], "tupl": [26, 47, 57, 61, 63, 73, 75, 92, 114, 128, 144, 153, 155, 161, 171, 178, 179, 180, 191, 196], "ndarrai": [26, 197], "properti": [27, 35, 44, 49, 51], "argument": [27, 47, 57, 73, 89, 130, 153, 179, 190, 194, 198], "elment": 51, "indices_or_sect": [52, 141], "nest": [56, 190, 191, 196], "correpsond": 56, "ddof": [58, 154], "equal_nan": 59, "nan": 59, "pad": [62, 63, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 160, 161], "dilat": [62, 63], "group": [62, 63, 164], "1d": [62, 64, 149], "convolut": [62, 63, 64, 160, 161], "channel": [62, 63, 160, 161], "c_in": [62, 63], "c_out": [62, 63], "featur": [62, 63, 164, 165, 166, 168, 170, 197], "convolv": [62, 63], "2d": 63, "spatial": [63, 164], "symmetr": 63, "discret": [64, 76, 77, 78, 79, 80, 81, 85, 86, 87, 162], "swap": 64, "conv": 64, "filter": [64, 160, 161, 191], "flip": 64, "signal": 64, "divis": 69, "quotient": 69, "mathrm": [71, 135], "frac": [71, 135, 164, 165, 168, 174], "pi": 71, "int_0": 71, "dx": 71, "erf": 72, "retain_graph": [73, 130], "node": [73, 156], "dict": [73, 95, 131, 191, 196], "leaf": [73, 178, 191], "preserv": [73, 128], "intend": 73, "control": [73, 194], "flow": [73, 145], "exponenti": 74, "insert": 75, "One": [76, 79, 85, 129], "fourier": [76, 77, 78, 79, 80, 81, 85, 86, 87], "truncat": [76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 125], "zero": [76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 159, 190], "dft": [76, 77, 78, 79, 80, 81, 85, 86, 87], "rfft": 82, "except": [82, 83, 85, 86, 87, 164, 197], "real": [82, 83, 84, 85, 86, 87], "rfft2": 83, "rfftn": 84, "silent": [85, 86, 87], "fun": [89, 92, 153, 155, 156], "cpp_function": [89, 153, 156], "variabl": [89, 92, 153, 155, 156], "strict": [90, 93, 191], "binari": [95, 130, 131, 132], "npy": [95, 130], "natur": [96, 98], "logarithm": [96, 97, 98, 99], "log": [98, 100, 102], "plu": 98, "exp": [100, 102, 119, 139], "stabl": [100, 102, 139], "multipl": [103, 109, 167], "prepend": 103, "remov": [103, 118, 144], "anoth": [103, 146, 157, 191], "negat": 110, "pad_width": 114, "constant_valu": 114, "edg": 114, "before_1": 114, "after_1": 114, "before_2": 114, "after_2": 114, "before_n": 114, "after_n": 114, "integ": [114, 117, 122, 141, 156, 162, 187], "before_i": 114, "after_i": 114, "extend": 114, "side": 114, "smaller": 115, "distribut": [117, 118, 119, 121, 125, 126], "prng": [117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 194], "num_sampl": 118, "unnorm": 118, "draw": 118, "uint32": [118, 187], "cdf": [119, 163, 181], "accord": [119, 157, 167], "seed": 120, "low": [122, 126], "high": [122, 126, 162, 190], "probabl": 122, "lower": [122, 125, 126], "upper": [122, 125, 126], "bound": [122, 125, 126, 163], "roadcast": 122, "domain": 125, "optino": 125, "uniformli": 126, "reciproc": 129, "arr": 130, "retain": 130, "dure": 130, "uncompress": 131, "my_path": 131, "tree_flatten": [131, 180], "transformerencod": 131, "128": [131, 190], "flat_param": 131, "keyword": [131, 132, 190, 194, 198], "compress": 132, "subarrai": 141, "being": [145, 190], "ident": 145, "prevent": 145, "unchang": [145, 170], "differ": [146, 197], "taken": 148, "prior": [148, 149], "equial": 148, "exclud": 149, "mse": 153, "param": [153, 190], "lvalu": 153, "dlvalu": 153, "dparam": 153, "lasso": 153, "l1": 153, "varianc": [154, 164], "divisor": 154, "cotang": 155, "in_ax": 156, "out_ax": 156, "prefix": [156, 178], "select": [157, 191], "in_channel": [160, 161], "out_channel": [160, 161], "kernel_s": [160, 161], "appli": [160, 161, 163, 164, 165, 166, 168, 169, 172, 175, 179, 181, 184, 185, 191], "nlc": 160, "learnabl": [160, 161, 171], "nhwc": 161, "height": 161, "width": 161, "num_embed": 162, "lookup": 162, "tabl": [162, 187], "typic": [162, 193], "usual": [162, 196], "vocabulari": 162, "approx": 163, "unit": [163, 169, 172, 181, 182, 183, 184, 185], "textrm": [163, 181], "phi": [163, 181], "geluapprox": 163, "sigma": [163, 172, 182, 183, 185], "60033": [163, 182], "0433603": [163, 182], "gelufast": 163, "773": [163, 183], "gelu_approx": [163, 181], "gelu_fast_approx": [163, 181], "regard": 163, "num_group": 164, "affin": [164, 165, 166], "pytorch_compat": 164, "var": [164, 165], "epsilon": [164, 165, 168, 174], "gamma": [164, 165, 168], "particular": 164, "split": 164, "preced": 164, "org": [164, 165, 168, 170], "1803": 164, "08494": 164, "stabil": [164, 165, 168], "1607": 165, "06450": 165, "query_input_dim": 167, "key_input_dim": 167, "value_input_dim": 167, "value_dim": 167, "value_output_dim": 167, "dot": [167, 178, 191], "attent": [167, 191], "head": 167, "aggreg": 167, "lineari": 167, "bias": [167, 191], "inf": 167, "neg": 167, "attend": 167, "1910": 168, "07467": 168, "rectifi": [169, 184], "rotat": 170, "consecut": 170, "larger": 170, "slightli": 170, "callabl": [171, 173, 178, 179, 191], "plain": 171, "cdot": [172, 182, 183, 185], "fn": [173, 179, 197], "wrt": 173, "whose": 173, "9": 174, "999": 174, "paper": 174, "omit": 174, "estim": 174, "m_": 174, "beta_1": 174, "m_t": 174, "g_t": [174, 177], "v_": [174, 177], "beta_2": 174, "v_t": [174, 177], "w_": [174, 177], "w_t": [174, 177], "lambda": [174, 177, 179, 191], "kingma": 174, "ba": 174, "2015": 174, "stochast": [174, 177], "iclr": 174, "basi": 175, "optimizerst": 175, "recurs": [176, 190, 191], "defaultdict": 176, "miss": 176, "contrast": 176, "present": 176, "momentum": 177, "descent": 177, "mu": 177, "strength": 177, "is_leaf": 178, "notat": [178, 191], "arbitrari": [178, 191], "depth": 178, "hello": [178, 180], "charact": 178, "flat": [178, 180], "everi": 179, "superset": 179, "extra": 179, "closer": 179, "dict_kei": 179, "main": [179, 190, 197], "recreat": 180, "world": 180, "42": 180, "faster": 181, "gelu": [182, 183], "exact": [182, 183], "0003": 182, "015": 183, "show": 187, "byte": 187, "bool_": 187, "uint8": 187, "unsign": 187, "uint16": 187, "16": [187, 191], "int8": 187, "sign": 187, "int16": 187, "int64": 187, "64": 187, "arm": 187, "arbitrarili": [190, 196, 197], "done": 190, "manual": 190, "explicitli": [190, 194], "solv": 190, "intuit": 190, "freez": [190, 191], "finetun": 190, "in_dim": [190, 191], "out_dim": [190, 191], "enumer": 190, "caus": 190, "local": 190, "scope": 190, "l2_loss": 190, "y_hat": 190, "trainable_paramet": [190, 191], "loss_and_grad": 190, "workhors": 190, "Its": 190, "frozen": [190, 191], "subset": [190, 191], "individu": 190, "action": 190, "preclud": 190, "pure": [190, 193], "pattern": 190, "achiev": 190, "other_input": 190, "necessari": 190, "wrap": 190, "subclass": 191, "concept": 191, "mymlp": 191, "in_proj": 191, "lazi": [191, 197], "map_fn": 191, "filter_fn": 191, "valid_parameter_filt": 191, "apply_to_modul": 191, "apply_fn": 191, "children": 191, "descend": 191, "filter_and_map": 191, "is_leaf_fn": 191, "content": 191, "found": 191, "whether": 191, "drop": 191, "idempot": 191, "ie": 191, "noop": 191, "unfreez": 191, "endswith": 191, "leaf_modul": 191, "load_weight": 191, "named_modul": 191, "save_weight": 191, "unfrozen": 191, "chang": 191, "tracer": 191, "partial": 191, "subsequ": 193, "implicit": 194, "fine": 194, "grain": 194, "manag": 194, "uniform": 194, "pseudo": 194, "altern": 194, "jax": [194, 197], "splittabl": 194, "threefri": 194, "counter": 194, "cycl": 196, "suppoort": 197, "cpu": 197, "gpu": 197, "strongli": 197, "inspir": 197, "arrayfir": 197, "noteabl": 197, "unifi": 197, "live": 197, "inspect": 197, "composit": 197, "sin": 197, "default_stream": 198, "default_devic": 198, "my_devic": 198}, "objects": {"mlx.core": [[7, 0, 1, "", "Device"], [8, 0, 1, "", "Dtype"], [9, 0, 1, "", "Stream"], [10, 2, 1, "", "abs"], [11, 2, 1, "", "add"], [12, 2, 1, "", "all"], [13, 2, 1, "", "allclose"], [14, 2, 1, "", "any"], [15, 2, 1, "", "arange"], [16, 2, 1, "", "arccos"], [17, 2, 1, "", "arccosh"], [18, 2, 1, "", "arcsin"], [19, 2, 1, "", "arcsinh"], [20, 2, 1, "", "arctan"], [21, 2, 1, "", "arctanh"], [22, 2, 1, "", "argmax"], [23, 2, 1, "", "argmin"], [24, 2, 1, "", "argpartition"], [25, 2, 1, "", "argsort"], [26, 0, 1, "", "array"], [59, 2, 1, "", "array_equal"], [60, 2, 1, "", "broadcast_to"], [61, 2, 1, "", "concatenate"], [62, 2, 1, "", "conv1d"], [63, 2, 1, "", "conv2d"], [64, 2, 1, "", "convolve"], [65, 2, 1, "", "cos"], [66, 2, 1, "", "cosh"], [67, 2, 1, "", "default_device"], [68, 2, 1, "", "default_stream"], [69, 2, 1, "", "divide"], [70, 2, 1, "", "equal"], [71, 2, 1, "", "erf"], [72, 2, 1, "", "erfinv"], [73, 2, 1, "", "eval"], [74, 2, 1, "", "exp"], [75, 2, 1, "", "expand_dims"], [88, 2, 1, "", "full"], [89, 2, 1, "", "grad"], [90, 2, 1, "", "greater"], [91, 2, 1, "", "greater_equal"], [92, 2, 1, "", "jvp"], [93, 2, 1, "", "less"], [94, 2, 1, "", "less_equal"], [95, 2, 1, "", "load"], [96, 2, 1, "", "log"], [97, 2, 1, "", "log10"], [98, 2, 1, "", "log1p"], [99, 2, 1, "", "log2"], [100, 2, 1, "", "logaddexp"], [101, 2, 1, "", "logical_not"], [102, 2, 1, "", "logsumexp"], [103, 2, 1, "", "matmul"], [104, 2, 1, "", "max"], [105, 2, 1, "", "maximum"], [106, 2, 1, "", "mean"], [107, 2, 1, "", "min"], [108, 2, 1, "", "minimum"], [109, 2, 1, "", "multiply"], [110, 2, 1, "", "negative"], [111, 2, 1, "", "new_stream"], [112, 2, 1, "", "ones"], [113, 2, 1, "", "ones_like"], [114, 2, 1, "", "pad"], [115, 2, 1, "", "partition"], [116, 2, 1, "", "prod"], [127, 2, 1, "", "reciprocal"], [128, 2, 1, "", "reshape"], [129, 2, 1, "", "rsqrt"], [130, 2, 1, "", "save"], [131, 2, 1, "", "savez"], [132, 2, 1, "", "savez_compressed"], [133, 2, 1, "", "set_default_device"], [134, 2, 1, "", "set_default_stream"], [135, 2, 1, "", "sigmoid"], [136, 2, 1, "", "sign"], [137, 2, 1, "", "sin"], [138, 2, 1, "", "sinh"], [139, 2, 1, "", "softmax"], [140, 2, 1, "", "sort"], [141, 2, 1, "", "split"], [142, 2, 1, "", "sqrt"], [143, 2, 1, "", "square"], [144, 2, 1, "", "squeeze"], [145, 2, 1, "", "stop_gradient"], [146, 2, 1, "", "subtract"], [147, 2, 1, "", "sum"], [148, 2, 1, "", "take"], [149, 2, 1, "", "take_along_axis"], [150, 2, 1, "", "tan"], [151, 2, 1, "", "tanh"], [152, 2, 1, "", "transpose"], [153, 2, 1, "", "value_and_grad"], [154, 2, 1, "", "var"], [155, 2, 1, "", "vjp"], [156, 2, 1, "", "vmap"], [157, 2, 1, "", "where"], [158, 2, 1, "", "zeros"], [159, 2, 1, "", "zeros_like"]], "mlx.core.Device": [[7, 1, 1, "", "__init__"]], "mlx.core.Dtype": [[8, 1, 1, "", "__init__"]], "mlx.core.Stream": [[9, 1, 1, "", "__init__"]], "mlx.core.array": [[27, 3, 1, "", "T"], [26, 1, 1, "", "__init__"], [28, 1, 1, "", "abs"], [29, 1, 1, "", "all"], [30, 1, 1, "", "any"], [31, 1, 1, "", "argmax"], [32, 1, 1, "", "argmin"], [33, 1, 1, "", "astype"], [34, 1, 1, "", "cos"], [35, 3, 1, "", "dtype"], [36, 1, 1, "", "exp"], [37, 1, 1, "", "item"], [38, 1, 1, "", "log"], [39, 1, 1, "", "log1p"], [40, 1, 1, "", "logsumexp"], [41, 1, 1, "", "max"], [42, 1, 1, "", "mean"], [43, 1, 1, "", "min"], [44, 3, 1, "", "ndim"], [45, 1, 1, "", "prod"], [46, 1, 1, "", "reciprocal"], [47, 1, 1, "", "reshape"], [48, 1, 1, "", "rsqrt"], [49, 3, 1, "", "shape"], [50, 1, 1, "", "sin"], [51, 3, 1, "", "size"], [52, 1, 1, "", "split"], [53, 1, 1, "", "sqrt"], [54, 1, 1, "", "square"], [55, 1, 1, "", "sum"], [56, 1, 1, "", "tolist"], [57, 1, 1, "", "transpose"], [58, 1, 1, "", "var"]], "mlx.core.fft": [[76, 2, 1, "", "fft"], [77, 2, 1, "", "fft2"], [78, 2, 1, "", "fftn"], [79, 2, 1, "", "ifft"], [80, 2, 1, "", "ifft2"], [81, 2, 1, "", "ifftn"], [82, 2, 1, "", "irfft"], [83, 2, 1, "", "irfft2"], [84, 2, 1, "", "irfftn"], [85, 2, 1, "", "rfft"], [86, 2, 1, "", "rfft2"], [87, 2, 1, "", "rfftn"]], "mlx.core.random": [[117, 2, 1, "", "bernoulli"], [118, 2, 1, "", "categorical"], [119, 2, 1, "", "gumbel"], [120, 2, 1, "", "key"], [121, 2, 1, "", "normal"], [122, 2, 1, "", "randint"], [123, 2, 1, "", "seed"], [124, 2, 1, "", "split"], [125, 2, 1, "", "truncated_normal"], [126, 2, 1, "", "uniform"]], "mlx.nn": [[160, 0, 1, "", "Conv1d"], [161, 0, 1, "", "Conv2d"], [162, 0, 1, "", "Embedding"], [163, 0, 1, "", "GELU"], [164, 0, 1, "", "GroupNorm"], [165, 0, 1, "", "LayerNorm"], [166, 0, 1, "", "Linear"], [191, 0, 1, "", "Module"], [167, 0, 1, "", "MultiHeadAttention"], [168, 0, 1, "", "RMSNorm"], [169, 0, 1, "", "ReLU"], [170, 0, 1, "", "RoPE"], [171, 0, 1, "", "Sequential"], [172, 0, 1, "", "SiLU"], [181, 0, 1, "", "gelu"], [182, 0, 1, "", "gelu_approx"], [183, 0, 1, "", "gelu_fast_approx"], [184, 0, 1, "", "relu"], [185, 0, 1, "", "silu"], [173, 2, 1, "", "value_and_grad"]], "mlx.nn.Module": [[191, 1, 1, "", "apply"], [191, 1, 1, "", "apply_to_modules"], [191, 1, 1, "", "children"], [191, 1, 1, "", "filter_and_map"], [191, 1, 1, "", "freeze"], [191, 1, 1, "", "leaf_modules"], [191, 1, 1, "", "load_weights"], [191, 1, 1, "", "modules"], [191, 1, 1, "", "named_modules"], [191, 1, 1, "", "parameters"], [191, 1, 1, "", "save_weights"], [191, 1, 1, "", "trainable_parameters"], [191, 1, 1, "", "unfreeze"], [191, 1, 1, "", "update"]], "mlx.optimizers": [[174, 0, 1, "", "Adam"], [175, 0, 1, "", "Optimizer"], [176, 0, 1, "", "OptimizerState"], [177, 0, 1, "", "SGD"]], "mlx.optimizers.Optimizer": [[175, 4, 1, "", "state"]], "mlx.utils": [[178, 2, 1, "", "tree_flatten"], [179, 2, 1, "", "tree_map"], [180, 2, 1, "", "tree_unflatten"]]}, "objtypes": {"0": "py:class", "1": "py:method", "2": "py:function", "3": "py:property", "4": "py:attribute"}, "objnames": {"0": ["py", "class", "Python class"], "1": ["py", "method", "Python method"], "2": ["py", "function", "Python function"], "3": ["py", "property", "Python property"], "4": ["py", "attribute", "Python attribute"]}, "titleterms": {"oper": [0, 1, 192], "develop": 1, "document": 1, "introduc": 1, "exampl": [1, 5], "primit": 1, "us": [1, 198], "implement": [1, 3], "cpu": 1, "backend": 1, "gpu": 1, "transform": [1, 195, 197], "build": [1, 6], "bind": 1, "python": [1, 5, 6], "cmake": 1, "setuptool": 1, "usag": [1, 5], "result": 1, "script": [1, 3], "download": [1, 3], "code": [1, 3], "linear": [2, 166], "regress": 2, "llm": 3, "infer": 3, "model": 3, "attent": 3, "layer": [3, 4, 190], "encod": 3, "full": [3, 88], "gener": 3, "put": 3, "all": [3, 12, 29], "togeth": 3, "convert": 3, "weight": 3, "load": [3, 95], "benchmark": 3, "multi": 4, "perceptron": 4, "mlx": [5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 191], "instal": [5, 6], "further": 5, "read": 5, "api": [5, 6], "refer": 5, "c": [5, 6], "from": 6, "pypi": 6, "sourc": 6, "requir": 6, "option": 6, "core": [7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159], "devic": [7, 188, 197], "dtype": [8, 35], "stream": [9, 188, 197, 198], "ab": [10, 28], "add": 11, "allclos": 13, "ani": [14, 30], "arang": 15, "arcco": 16, "arccosh": 17, "arcsin": 18, "arcsinh": 19, "arctan": 20, "arctanh": 21, "argmax": [22, 31], "argmin": [23, 32], "argpartit": 24, "argsort": 25, "arrai": [26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 186], "t": 27, "astyp": 33, "co": [34, 65], "exp": [36, 74], "item": 37, "log": [38, 96], "log1p": [39, 98], "logsumexp": [40, 102], "max": [41, 104], "mean": [42, 106], "min": [43, 107], "ndim": 44, "prod": [45, 116], "reciproc": [46, 127], "reshap": [47, 128], "rsqrt": [48, 129], "shape": 49, "sin": [50, 137], "size": 51, "split": [52, 124, 141], "sqrt": [53, 142], "squar": [54, 143], "sum": [55, 147], "tolist": 56, "transpos": [57, 152], "var": [58, 154], "array_equ": 59, "broadcast_to": 60, "concaten": 61, "conv1d": [62, 160], "conv2d": [63, 161], "convolv": 64, "cosh": 66, "default_devic": 67, "default_stream": 68, "divid": 69, "equal": 70, "erf": 71, "erfinv": 72, "eval": 73, "expand_dim": 75, "fft": [76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 189], "fft2": 77, "fftn": 78, "ifft": 79, "ifft2": 80, "ifftn": 81, "irfft": 82, "irfft2": 83, "irfftn": 84, "rfft": 85, "rfft2": 86, "rfftn": 87, "grad": [89, 190], "greater": 90, "greater_equ": 91, "jvp": 92, "less": 93, "less_equ": 94, "log10": 97, "log2": 99, "logaddexp": 100, "logical_not": 101, "matmul": 103, "maximum": 105, "minimum": 108, "multipli": 109, "neg": 110, "new_stream": 111, "ones": 112, "ones_lik": 113, "pad": 114, "partit": 115, "random": [117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 194], "bernoulli": 117, "categor": 118, "gumbel": 119, "kei": 120, "normal": 121, "randint": 122, "seed": 123, "truncated_norm": 125, "uniform": 126, "save": 130, "savez": 131, "savez_compress": 132, "set_default_devic": 133, "set_default_stream": 134, "sigmoid": 135, "sign": 136, "sinh": 138, "softmax": 139, "sort": 140, "squeez": 144, "stop_gradi": 145, "subtract": 146, "take": 148, "take_along_axi": 149, "tan": 150, "tanh": 151, "value_and_grad": [153, 173], "vjp": 155, "vmap": 156, "where": 157, "zero": 158, "zeros_lik": 159, "nn": [160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 181, 182, 183, 184, 185, 191], "embed": 162, "gelu": [163, 181], "groupnorm": 164, "layernorm": 165, "multiheadattent": 167, "rmsnorm": 168, "relu": [169, 184], "rope": 170, "sequenti": 171, "silu": [172, 185], "optim": [174, 175, 176, 177, 193], "adam": 174, "optimizerst": 176, "sgd": 177, "util": [178, 179, 180, 196], "tree_flatten": 178, "tree_map": 179, "tree_unflatten": 180, "gelu_approx": 182, "gelu_fast_approx": 183, "data": 187, "type": 187, "support": 187, "neural": 190, "network": 190, "quick": [190, 197], "start": [190, 197], "The": 190, "modul": [190, 191], "class": 190, "paramet": 190, "updat": 190, "valu": 190, "tree": 196, "guid": 197, "basic": 197, "function": 197, "graph": 197, "specifi": 198}, "envversion": {"sphinx.domains.c": 2, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 6, "sphinx.domains.index": 1, "sphinx.domains.javascript": 2, "sphinx.domains.math": 2, "sphinx.domains.python": 3, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "sphinx.ext.intersphinx": 1, "sphinx": 56}}) \ No newline at end of file diff --git a/docs/build/html/using_streams.html b/docs/build/html/using_streams.html new file mode 100644 index 000000000..091f19bfe --- /dev/null +++ b/docs/build/html/using_streams.html @@ -0,0 +1,157 @@ + + + + + + + Using Streams — MLX 0.0.0 documentation + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Using Streams

+
+

Specifying the Stream

+

All operations (including random number generation) take an optional +keyword argument stream. The stream kwarg specifies which +Stream the operation should run on. If the stream is unspecified then +the operation is run on the default stream of the default device: +mx.default_stream(mx.default_device()). The stream kwarg can also +be a Device (e.g. stream=my_device) in which case the operation is +run on the default stream of the provided device +mx.default_stream(my_device).

+
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file