Misplaced Pages

#481518

43-617: HBM was adopted by JEDEC as an industry standard in October 2013. The second generation, HBM2 , was accepted by JEDEC in January 2016. JEDEC officially announced the HBM3 standard on January 27, 2022. HBM achieves higher bandwidth than DDR4 or GDDR5 while using less power, and in a substantially smaller form factor. This is achieved by stacking up to eight DRAM dies and an optional base die which can include buffer circuitry and test logic. The stack

86-511: A clock signal running at 100 MHz. Buses like SCSI and PCI fall in the megatransfer range of data transfer rate, while newer bus architectures like the PCI-X , PCI Express , Ultra Path , and HyperTransport / Infinity Fabric operate at the gigatransfer rate. The choice of the symbol T for transfer conflicts with the International System of Units , in which T is the symbol for

129-507: A free registration. JEDEC has issued widely used standards for device interfaces, such as the JEDEC memory standards for computer memory ( RAM ), including the DDR SDRAM standards. JEDEC also developed a number of popular package drawings for semiconductors such as TO-3 , TO-5 , etc. These are on the web under JEP-95. One hot issue is the development of lead-free packages that do not suffer from

172-963: A future and faster one. JEDEC officially announced the HBM3 standard on January 27, 2022. The number of memory channels was doubled from 8 channels of 128 bits with HBM2e to 16 channels of 64 bits with HBM3. Therefore, the total number of data pins of the interface is still 1024. In June 2022, SK Hynix announced they started mass production of industry's first HBM3 memory to be used with Nvidia's H100 GPU expected to ship in Q3 2022. The memory will provide H100 with "up to 819 GB/s" of memory bandwidth. In August 2022, Nvidia announced that its "Hopper" H100 GPU will ship with five active HBM3 sites (out of six on board) offering 80 GB of RAM and 3 TB/s of memory bandwidth (16 GB and 600 GB/s per site). On 30 May 2023, SK Hynix unveiled its HBM3E memory with 8 Gbps/pin data processing speed (25% faster than HBM3), which

215-517: A graphics card with a 512‑bit memory interface. HBM supports up to 4 GB per package. The larger number of connections to the memory, relative to DDR4 or GDDR5, required a new method of connecting the HBM memory to the GPU (or other processor). AMD and Nvidia have both used purpose-built silicon chips, called interposers , to connect the memory and GPU. This interposer has the added advantage of requiring

258-421: A lower-cost version of HBM under development targeting mass markets. Removing the buffer die and decreasing the number of TSVs lowers cost, though at the expense of a decreased overall bandwidth (200 GB/s). Nvidia announced Nvidia Hopper H100 GPU, the world's first GPU utilizing HBM3 on March 22, 2022. JEDEC The JEDEC [Joint Electron Device Engineering Council] Solid State Technology Association

301-447: A patent may be adopted, but only on the understanding that the patent owner will not enforce such patent rights or, at a minimum, that the patent owner will provide a reasonable and non-discriminatory license to the patented technology. JEDEC's early work began as a part numbering system for devices which became popular in the 1960s. The first semiconductor devices, such as the 1N23 silicon point contact diode, were still designated in

344-497: A similar way from the older Mullard–Philips tube designation . This early work was followed by a number of test methods, JESD22, and product standards. For example, the ESD caution symbol, which is the hand with the line drawn through it, was published by JEDEC and is used worldwide. JEDEC also has a dictionary of semiconductor terms. All of JEDEC standards are free on the Web for downloading after

387-432: A team led by Senior AMD Fellow Bryan Black. To help AMD realize their vision of HBM, they enlisted partners from the memory industry, particularly Korean company SK Hynix , which had prior experience with 3D-stacked memory, as well as partners from the interposer industry (Taiwanese company UMC ) and packaging industry ( Amkor Technology and ASE ). The development of HBM was completed in 2013, when SK Hynix built

430-502: A transfer rate of 3.2  GT/s , providing a total of 16 GB and 410 GB/s per stack. August 12, 2019, SK Hynix announced their HBM2E, featuring eight dies per stack, a transfer rate of 3.6  GT/s , providing a total of 16 GB and 460 GB/s per stack. On July 2, 2020, SK Hynix announced that mass production has begun. In late 2020, Micron unveiled that the HBM2E standard would be updated and alongside that they unveiled

473-752: A type number that was assigned by JETEC. In the fall of 1999, JEDEC became a separate trade association under the current name, but maintained an EIA alliance. JEDEC has adopted the principle of open standards , which permit any and all interested companies to freely manufacture in compliance with adopted standards. This serves several vital functions for the advancement of electronic technologies. First and foremost, such standards allow for interoperability between different electrical components. JEDEC standards do not protect members from normal patent obligations. The designated representatives of JEDEC member companies are required to disclose patents and patent applications of which they are aware, assuming that this information

SECTION 10

#1732851624482

516-773: A wide-interface architecture to achieve high-speed, low-power operation. The HBM DRAM uses a 500 MHz differential clock CK_t / CK_c (where the suffix "_t" denotes the "true", or "positive", component of the differential pair, and "_c" stands for the "complementary" one). Commands are registered at the rising edge of CK_t, CK_c. Each channel interface maintains a 128‑bit data bus operating at double data rate (DDR). HBM supports transfer rates of 1  GT/s per pin (transferring 1 bit), yielding an overall package bandwidth of 128 GB/s. The second generation of High Bandwidth Memory, HBM2, also specifies up to eight dies per stack and doubles pin transfer rates up to 2  GT/s . Retaining 1024‑bit wide access, HBM2

559-520: Is 64 Gbit/s (8 × 8 × 10  bit/s). The formula for a data transfer rate is: Channel width (bits/transfer) × transfers/second = bits/second . Expanding the width of a channel, for example that between a CPU and a northbridge , increases data throughput without requiring an increase in the channel's operating frequency (measured in transfers per second). This is analogous to increasing throughput by increasing bandwidth but leaving latency unchanged. The units usually refer to

602-503: Is able to reach 256 GB/s memory bandwidth per package. The HBM2 spec allows up to 8 GB per package. HBM2 is predicted to be especially useful for performance-sensitive consumer applications such as virtual reality . On January 19, 2016, Samsung announced early mass production of HBM2, at up to 8 GB per stack. SK Hynix also announced availability of 4 GB stacks in August 2016. In late 2018, JEDEC announced an update to

645-563: Is an independent semiconductor engineering trade organization and standardization body headquartered in the United States. JEDEC has over 300 members, including some of the world's largest computer companies. Its scope and past activities includes standardization of part numbers , defining an electrostatic discharge (ESD) standard, and leadership in the lead-free manufacturing transition. The origin of JEDEC traces back to 1944, when RMA (subsequently renamed EIA ) and NEMA established

688-500: Is likely to be used mostly in data center GPUs . In mid 2021, SK Hynix unveiled some specifications of the HBM3 standard, with 5.2 Gbit/s I/O speeds and bandwidth of 665 GB/s per package, as well as up to 16-high 2.5D and 3D solutions. On 20 October 2021, before the JEDEC standard for HBM3 was finalised, SK Hynix was the first memory vendor to announce that it has finished development of HBM3 memory devices. According to SK Hynix,

731-405: Is not considered proprietary. JEDEC patent policy requires that standards found to contain patented technology, whose owners do not sign a standard JEDEC patent letter, be withdrawn. Thus the penalty for a failure to disclose patents is retraction of the standard. Typically, standards are not adopted to cover technology that are subject to patent protection. In rare circumstances, standards covered by

774-566: Is often connected to the memory controller on a GPU or CPU through a substrate, such as a silicon interposer . Alternatively, the memory die could be stacked directly on the CPU or GPU chip. Within the stack the dies are vertically interconnected by through-silicon vias (TSVs) and microbumps . The HBM technology is similar in principle but incompatible with the Hybrid Memory Cube (HMC) interface developed by Micron Technology . HBM memory bus

817-402: Is to enter production in the first half of 2024. At 8 GT/s with 1024-bit bus, its bandwidth per stack is increased from 819.2 GB/s as in HBM3 to 1 TB/s. On 26 July 2023, Micron announced its HBM3E memory with 9.6 Gbps/pin data processing speed (50% faster than HBM3). Micron HBM3E memory is a high-performance HBM that uses 1β DRAM process technology and advanced packaging to achieve

860-436: Is very wide in comparison to other DRAM memories such as DDR4 or GDDR5. An HBM stack of four DRAM dies (4‑Hi) has two 128‑bit channels per die for a total of 8 channels and a width of 1024 bits in total. A graphics card/GPU with four 4‑Hi HBM stacks would therefore have a memory bus with a width of 4096 bits. In comparison, the bus width of GDDR memories is 32 bits, with 16 channels for

903-667: The Radio Manufacturers Association (RMA), and the National Electrical Manufacturers Association (NEMA) established the Joint Electron Tube Engineering Council ( JETEC ) to coordinate vacuum tube type numberings. The expansion of the radio industry caused JETEC to expand its scope to include solid-state devices and develop standards for semiconductor devices . Eventually, the joint JETEC activity of EIA and NEMA

SECTION 20

#1732851624482

946-493: The tin whiskers problem that reappeared since the recent ban on lead content . JEDEC is working with iNEMI on a joint interest group on lead-free issues. As of 2023, JEDEC has 365 members in total. Among them are large companies, which include the following. GT/s In computer technology, transfers per second and its more common secondary terms gigatransfers per second (abbreviated as GT/s ) and megatransfers per second ( MT/s ) are informal language that refer to

989-498: The "effective" number of transfers, or transfers perceived from "outside" of a system or component, as opposed to the internal speed or rate of the clock of the system. One example is a computer bus running at double data rate where data is transferred on both the rising and falling edge of the clock signal. If its internal clock runs at 100 MHz, then the effective rate is 200 MT/s, because there are 100 million rising edges per second and 100 million falling edges per second of

1032-536: The HBM2 specification, providing for increased bandwidth and capacities. Up to 307 GB/s per stack (2.5 Tbit/s effective data rate) is now supported in the official specification, though products operating at this speed had already been available. Additionally, the update added support for 12‑Hi stacks (12 dies) making capacities of up to 24 GB per stack possible. On March 20, 2019, Samsung announced their Flashbolt HBM2E, featuring eight dies per stack,

1075-478: The HBM3 version. In May 2023, Samsung announced HBM3P with up to 7.2 Gbps which will be in production in 2024. On October 20 2023, Samsung announced their HBM3E "Shinebolt" with up to 9.8 Gbps memory. On February 26 2024, Micron announced the mass production of Micron's HBM3E memory. On March 18 2024, Nvidia announced the Blackwell series of GPUs using HBM3E memory On March 19 2024, SK Hynix announced

1118-573: The JESD229 standard for Wide IO memory, the predecessor of HBM featuring four 128 bit channels with single data rate clocking, in December 2011 after several years of work. The first HBM standard JESD235 followed in October 2013. The development of High Bandwidth Memory began at AMD in 2008 to solve the problem of ever-increasing power usage and form factor of computer memory. Over the next several years, AMD developed procedures to solve die-stacking problems with

1161-598: The Joint Electron Tube Engineering Council (JETEC) to coordinate vacuum tube type numberings . In 1958, with the advent of semiconductor technology, the joint JETEC-activity of EIA and NEMA was renamed into Joint Electron Device Engineering Council . NEMA discontinued its involvement in 1979. In the fall of 1999, JEDEC became a separate trade association under the current name, but maintained an EIA alliance, until EIA ceased operations in 2011. The origin of JEDEC can be traced back to 1944, when

1204-877: The SK Hynix first generation HBM3 memory has the same density as their latest-generation HBM2E memory, meaning that device vendors looking to increase their total memory capacities for their next-generation parts would need to use memory with 12 dies/layers, up from the 8 layer stacks they typically used until then. According to Anton Shilov of Tom's Hardware , high-performance compute GPUs or FPGAs typically use four or six HBM stacks, so with SK Hynix's HBM3 24 GB stacks they would accordingly get 3.2 TB/s or 4.9 TB/s of memory bandwidth. He also noted that SK Hynix's HBM3 chips are square, not rectangular like HBM2 and HBM2E chips. According to Chris Mellor of The Register , with JEDEC not yet having developed its HBM3 standard, might mean that SK Hynix would need to retrofit its design to

1247-450: The channel width or word length ). In order to calculate the data transmission rate, one must multiply the transfer rate by the information channel width. For example, a data bus eight-bytes wide (64 bits) by definition transfers eight bytes in each transfer operation; at a transfer rate of 1 GT/s, the data rate would be 8 × 10   B /s, i.e. 8 GB/s, or approximately 7.45  GiB /s. The bit rate for this example

1290-446: The context of transmission of digital data. 1 MT/s is 10 or one million transfers per second; similarly, 1 GT/s means 10 , or equivalently in the US/ short scale , one billion transfers per second. These terms alone do not specify the bit rate at which binary data is being transferred because they do not specify the number of bits transferred in each transfer operation (known as

1333-671: The first HBM memory chip. HBM was adopted as industry standard JESD235 by JEDEC in October 2013, following a proposal by AMD and SK Hynix in 2010. High volume manufacturing began at a Hynix facility in Icheon , South Korea, in 2015. The first GPU utilizing HBM was the AMD Fiji which was released in June 2015 powering the AMD Radeon R9 Fury ;X. In January 2016, Samsung Electronics began early mass production of HBM2. The same month, HBM2

High Bandwidth Memory - Misplaced Pages Continue

1376-457: The highest performance, capacity and power efficiency in the industry. It can store 24 GB per 8-high cube and allows data transfer at 1.2 TB/s. There will be a 12-high cube with 36 GB capacity in 2024. In August 2023, Nvidia announced a new version of their GH200 Grace Hopper superchip that utilizes 141 GB (144 GiB physical) of HBM3e over a 6144-bit bus providing 50% higher memory bandwidth and 75% higher memory capacity over

1419-417: The mass production of SK Hynix's HBM3E memory. In February 2021, Samsung announced the development of HBM with processing-in-memory (PIM). This new memory brings AI computing capabilities inside the memory, to increase the large-scale processing of data. A DRAM-optimised AI engine is placed inside each memory bank to enable parallel processing and minimise data movement. Samsung claims this will deliver twice

1462-495: The memory and processor to be physically close, decreasing memory paths. However, as semiconductor device fabrication is significantly more expensive than printed circuit board manufacture, this adds cost to the final product. The HBM DRAM is tightly coupled to the host compute die with a distributed interface. The interface is divided into independent channels. The channels are completely independent of one another and are not necessarily synchronous to each other. The HBM DRAM uses

1505-744: The memory would run as fast as 6.4 Gbps/pin, double the data rate of JEDEC-standard HBM2E, which formally tops out at 3.2 Gbps/pin, or 78% faster than SK Hynix’s own 3.6 Gbps/pin HBM2E. The devices support a data transfer rate of 6.4 Gbit/s and therefore a single HBM3 stack may provide a bandwidth of up to 819 GB/s. The basic bus widths for HBM3 remain unchanged, with a single stack of memory being 1024-bits wide. SK Hynix would offer their memory in two capacities: 16 GB and 24 GB, aligning with 8-Hi and 12-Hi stacks respectively. The stacks consist of 8 or 12 16 Gb DRAMs that are each 30 μm thick and interconnected using Through Silicon Vias (TSVs). According to Ryan Smith of AnandTech ,

1548-462: The next standard known as HBMnext (later renamed to HBM3). This was to be a big generational leap from HBM2 and the replacement to HBM2E. This new VRAM would have come to the market in the Q4 of 2022. This would likely introduce a new architecture as the naming suggests. While the architecture might be overhauled, leaks point toward the performance to be similar to that of the updated HBM2E standard. This RAM

1591-402: The number of operations transferring data that occur in each second in some given data-transfer channel . It is also known as sample rate , i.e. the number of data samples captured per second, each sample normally occurring at the clock edge. The terms are neutral with respect to the method of physically accomplishing each such data-transfer operation; nevertheless, they are most commonly used in

1634-447: The old RMA tube designation system, where the "1" stood for "No filament/heater" and the "N" stood for "crystal rectifier". The first RMA digit thus was re-allocated from "heater power" to "p-n junction count" to form the new EIA/JEDEC EIA-370 standard; for example, the 1N4001 rectifier diode and 2N2222 transistor part numbers came from EIA-370 . They are still popular today. In February 1982, JEDEC issued JESD370B , superseding

1677-431: The original EIA-370 and introducing a new letter symbol "C" that denotes the die version , as opposed to "N", now meaning the packaged version . The Japanese JIS semiconductor designation system employs a similar pattern. JEDEC later developed a numbering system for integrated circuits, but this did not gain acceptance in the semiconductor industry. The European Pro Electron semiconductor numbering system originated in

1720-612: The system performance and reduce energy consumption by more than 70%, while not requiring any hardware or software changes to the rest of the system. Die-stacked memory was initially commercialized in the flash memory industry. Toshiba introduced a NAND flash memory chip with eight stacked dies in April 2007, followed by Hynix Semiconductor introducing a NAND flash chip with 24 stacked dies in September 2007. 3D-stacked random-access memory (RAM) using through-silicon via (TSV) technology

1763-592: Was accepted by JEDEC as standard JESD235a. The first GPU chip utilizing HBM2 is the Nvidia Tesla P100 which was officially announced in April 2016. In June 2016, Intel released a family of Xeon Phi processors with 8 stacks of HCDRAM, Micron's version of HBM. At Hot Chips in August 2016, both Samsung and Hynix announced a new generation HBM memory technologies. Both companies announced high performance products expected to have increased density, increased bandwidth, and lower power consumption. Samsung also announced

High Bandwidth Memory - Misplaced Pages Continue

1806-528: Was commercialized by Elpida Memory , which developed the first 8   GB DRAM chip (stacked with four DDR3 SDRAM dies) in September 2009, and released it in June 2011. In 2011, SK Hynix introduced 16   GB DDR3 memory ( 40   nm class) using TSV technology, Samsung Electronics introduced 3D-stacked 32   GB DDR3 ( 30   nm class) based on TSV in September, and then Samsung and Micron Technology announced TSV-based Hybrid Memory Cube (HMC) technology in October. JEDEC first released

1849-432: Was renamed into Joint Electron Device Engineering Council ( JEDEC ) in 1958. NEMA discontinued its involvement in 1979. Earlier in the 20th century, the organization was known as JETEC, the Joint Electron Tube Engineering Council, and was responsible for assigning and coordinating RETMA tube designations to electron tubes (also called valves). The type 6L6 , still to be found in electric-guitar amplifiers, typically has

#481518